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/'This  first  volume  of  the  handbook  presents  procedures  that  provide  a 
structured  approach  for  estimating  computer  system  sizing  and  timing 
parameters  during  the  acquisition  life-cycle  of  Embedded  Computer  Systems 
(ECS)  .  The  procedures  were  designed  for  use  by  the  Electronic  Systems 
Division's  Comput  -  Systems  Engineering  Directorate  (ESD/TOI).  Based  on 
engineering  discipline,  the  procedures  provide  a  step-by-step  program  to 
assist  ESD/TOI  engineers  and  computer  scientists  in  evaluating  or 
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^conducting  initial  sizing  and  timing  estimates,  updates,  or  developmental 
monitoring.  Procedural  steps  are  discussed  with  emphasis  on  actions  that 
should  be  taken,  risks  that  should  be  considered,  constraints  that  may  be 
encountered,  and  factors  that  may  affect  the  quality  of  the  estimate  at 
various  stages  of  the  acquisition.  Techniques  that  can  be  used  in 

estimating  sizing  and  timing  parameters  are  also  discussed,  including  data 
requirements,  assumptions,  and  levels  of  confidence. 

In  addition,  this  volume  also  contains  a  reference  listing,  bibliography, 
glossary  and  an  index. 

The  second  volume  of  the  handbook  is  an  addendum  that  contains 

supplemental  illustrative  information  regarding  selected  ESD  C3  Embedded 
Computer  Systems  (ECS),  sets  of  graphic  representations  of  sizing  and 

timing  relationships  of  computer  systems,  two  examples  of  the  application 
of  the  procedures  presented  in  this  volume,  a  proposed  revision  of  the 
Data  Item  Description  (DID)  entitled,  Computer  Program  Timing  and  Sizing 
Data  (DI-S-30568 ) ,  a  list  of  suggestions  for  future  work  to  update  and 
improve  the  estimating  procedures,  a  reference  listing,  an  annotated 

bibliography,  and  an  index. 
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PREFACE 


This  handbook  of  procedures  for  estimating  computer  system  sizing  and 
timing  parameters  during  the  acquisition  life-cycle  of  Embedded  Computer 
Systems  (ECS)  was  developed  for  use  by  the  Electronic  Systems  Division's 
Computer  Systems  Engineering  Directorate  (ESD/TOI).  Based  on  engineering 
discipline,  ‘‘he  procedures  provide  a  step-by-step  program  to  assist 
ESD/TOI  engineers  and  computer  scientists  in  evaluating  or  conducting 
initial  sizing  and  timing  estimates,  updates,  or  developmental 
monitoring.  Procedural  steps  are  discussed  with  emphasis  on  actions  that 
should  be  taken,  risks  that  should  be  considered,  constraints  that  may  be 
encountered,  and  factors  that  may  affect  the  quality  of  the  estimate  at 
various  stages  of  the  acquisition.  An  introduction  to  techniques  that  can 
be  used  in  estimating  sizing  and  timing  parameters  is  presented,  including 
data  requirements,  assumptions,  and  levels  of  confidence. 


The  handbook  consists  of  two  volumes;  this  first  volume  discusses 
procedures  and  techniques,  and  the  second  volume  provides  an  addendum  of 
supplemental  information.  The  primary  features  and  tools  of  this  handbook 
are  as  follows; 
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INTRODUCTION 


1 . 1  Background. 

For  each  new  generation  of  Command,  Control,  and  Communications  (C3) 
systems,  Embedded  Computer  Systems  (ECS'  become  increasingly  critical. 
There  are  many  reasons  for  this;  computer  systems,  including  software,  are 
performing  more  functions  more  rapidly,  and  they  must  be  reliable  under 
more  extreme  physical  and  operational  constraints.  As  more  stringent 
requirements  have  been  imposed  upon  Command,  Control,  and  Communicat ions 
(C3)  system  performance  (e.g.,  ito..  targets  and  target  types,  more 
sophisticated  signal  processing,  improved  countermeasures  and  counter¬ 
countermeasures,  faster  responses,  and  greater  reliability),  increasing 
demands  have  been  placed  upon  the  C^  computer  systems. 

In  many  system  acquisitions,  the  uncertainties  of  qualifying  and 
quantifying  the  hardware  and  software  needs  of  an  ECS  have  been  extensive, 
with  software  uncertainties  far  exceeding  those  of  the  hardware.  This 
situation  is  projected  to  increase  in  frequency  and  severity  in  the 
future.  The  requirement  to  perform  most  necessary  functions  in  real-time 
is  very  demanding  and  has  thus  provided  the  impetus  for  continued 
improvement  in  hardware  and  software  technologies.  However,  in  spite  of 
these  technological  improvements,  increased  operational  requirements 
continue  to  dictate  larger  and  more  sophisticated  computer  systems  and 
software  packages.  As  a  consequence,  there  is  an  increasing,  if  not 
urgent,  need  to  develop  and  evaluate  embedded  computer  systems  sizing  and 
timing  estimates. 

The  field  of  computer  system  acquisition  management  and  engineering  is 
just  developing,  especially  in  its  ability  to  derive  sizing  and  timing 
estimates  for  entire  computer  systems  and  software  in  the  overall  computer 
system  environment.  Sizing  and  timing  of  whole  or  large  segments  of 
computer  systems  are  particularly  complex  and,  at  the  present  state-of- 
the-art,  only  appear  feasible  using  models,  simulations,  benchmarks  or 
monitors.  The  primary  difficulty  is  the  definition  of  a  workload  that  a 


computer  system  or  subsystem  must  perform.  Additionally,  the  following 
difficulties  in  sizing  and  timing  computer  systems  have  been  acknowledged: 

•  lack  of  standard  engineering  and  management  procedures 
and  techniques, 

•  lack  of  standard  metrics  used  in  developing  estimates, 

•  lack  of  understanding  of  the  relationships  between 
hardware  and  software  characteristics  and  operational 
requirements, 

•  lack  of  accurate  or  timely  projections  of  hardware  and 
software  resource  requirements,  and 

•  lack  of  awareness  of  viable  system  design  and  techno¬ 
logical  advances. 

The  Department  of  Defense,  recognizing  these  deficiencies,  established 
the  DoD  Management  Steering  Committee  for  embedded  computer  resources.  As 
stated  in  the  background  section  of  the  charter  for  the  group  in  DoD 
Directive  5000.29: 

..Current  annual  expenditures  by  the  Department  of 
Defense  on  the  design,  development,  acquisition, 
management  and  operation  support  of  computer  resources 
embedded  within  and  integral  to  weapons,  communi¬ 
cations,  command  and  control,  and  intelligence  systems 
are  measured  in  the  billions  of  dollars.  At  the  same 
time,  such  computer  resources  have  often  presented 
critical  cost  and  schedule  problems  during  the  develop¬ 
ment  and  acquisition  of  new  defense  systems.  Even 
after  system  implementation  and  fielding,  the  software 
has  often  proven  unreliable... 

Computer  system  resource  estimating  has  historically  been  charac¬ 
terized  by  two  shor tcomings;  it  has  been  poorly  done  and  seldom  vali¬ 
dated.  The  reasons  for  poor  estimating  are  numerous.  The  lack  of 
necessary  information  resources  to  implement  reliable  estimating 
methodologies  is  among  the  most  important;  also: 

•  There  is  no  common  data  base  from  which  to  develop 
computer  resource  estimates. 


2 


•  Even  where  some  historical  computer  systems  data  exist, 
there  is  often  no  clear  understanding  of  what  the  data 
actually  represent. 

The  problems  associated  with  embedded  computer  system  sizing  and 
timing  estimates  are  more  pronounced  and  diverse  than  those  associated 
with  stand-alone  systems.  When  a  C3  system  is  developed  with  integrated 
embedded  computers,  there  is  often  concurrent  hardware/sof tware  develop¬ 
ment.  An  immediate  problem,  therefore,  is  that  there  may  be  no  function¬ 
ing  hardware  on  which  to  begin  software  integration.  The  concepts  of 
early  defect  removal  using  modern  programming  practices  may  be  less  than 
adequate  to  prevent  severe  software  or  hardware  problems  when  the  system 
is  completed.  Estimates  can  be  done  by  use  of  models,  simulations, 
benchmarks  or  monitoring;  however,  these  techniques  may  not  be  feasible 
due  to  costs,  time  constraints,  or  lack  of  relevant  data. 

The  above  discussion  of  the  background  highlights  some  of  the  major 
problems  and  concerns  regarding  estimation  of  computer  system  sizing  and 
timing  parameters.  It  also  demonstrates  the  critical  need  for  Ait  Force 
managers  to  acquire  standardized  engineering  and  management  procedures, 
now  presented  in  this  handbook,  to  insure  that  the  operational 
requirements  of  ECS  are  met  during  the  system  acquisition  life-cycle 
in  the  most  efficient  and  effective  manner. 

1.2  Goal  and  Content. 

The  main  emphasis  of  this  handbook  is  to  provide  standard  procedures 
rather  than  metrics  for  estimating  sizing  and  timing  parameters  of  ECS  in 
Air  Force  C-3  systems,  for  use  by  the  engineers  and  computer  scientists 
of  the  Electronic  Systems  Division's  Computer  Systems  Engineering 
Directorate  (ESD/TOI).  Such  use  should  enable  the  engineers  and  computer 
scientists  of  ESD/TOI  to  better  assist  other  personnel  at  ESD  in  con¬ 
ducting  sizing  and  timing  studies.  IN  THE  EVENT  ESD  PERSONNEL  NEED 


ASSISTANCE  IN  CONDUCTING  A  SIZING  AND  TIMING  STUDY  -  CONTACT  ESD/TOI. 


r-: .  ;  .3  ••ir«>r>k  ccns  i.*ts  of  t  >  volume.'.  ~h : .~  f- I  ,;ts  ■cr'j'n-  *• 
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*  r.c  unct.vR  o'  estimating  ECS  giving  and  timing  parameters  ;  »e 

’-/pec:  initial  studies,  study  updates,  and  developmental  monitoring,  and 

provides  <-  step-by-step  program  employing  many  of  the  techniques  used  in 
Ver if icaion  and  Validation  (V&V)  of  systems.  The  procedural  steps  are 
discussed  with  emphasis  on  actions  that  should  be  taken,  risks  that  should 
be  considered,  constraints  that  may  be  encountered,  and  factors  that 
affect  the  quality  of  an  estimate  at  various  stages  of  the  acquisition. 
Techniques  that  can  be  used  in  estimating  sizing  and  timing  parameters  at 
various  stages  of  the  acquisition  are  discussed,  and  include  general 
information  regarding  required  data,  assumptions,  application,  and  level 
of  confidence.  In  the  second  volume,  information  about  selected  ESD  C3 
ECS  is  presented  to  illustrate  the  types  of  data  that  could  be  considered 
in  the  development  of  computer  system  analogies  for  sizing  and  timing 
studies.  Also  included  are  a  number  of  graphically  illustrated  computer 
system  sizing  and  timing  relative  relationships  that  are  based  on 
algorithms  developed  by  various  computer  scientists.  In  addition,  there 
are  two  examples  that  discuss  the  application  of  the  procedures. 

1 . 3  Organization  of  this  Volume. 

The  organization  of  the  remainder  of  this  volume  is  as  follows: 
Section  2  discusses  the  acquisition  life-cycle  of  ECS;  Section  3  discusses 
the  establishment  of  ECS  requirements;  Section  4  provides  an  overview  of 
the  development  of  ECS  sizing  and  timing  estimates;  Section  5  presents  the 
step-by-step  procedures  for  conducting  sizing  and  timing  studies;  Section 
6  discusses  the  integration  of  sizing  and  timing;  Tab  A  provides  infor¬ 
mation  on  sizing  and  timing  techniques;  and  in  addition,  there  is  a 
reference  listing,  bibliography,  glossary,  and  an  index  for  use  with  this 
handbook. 
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:.L  .■r.r.r-y^-iCtiori. 

In  order  to  address  the  problems  and  solutions  associated  v>h  the 
est  :3ia*.  ion  of  ECS  sizing  and  timing  parameters,  it  is  helpful  to  review 
the  acquisition  life-cycle.  'rhe  process  of  system  acquisition  has  evolved 

over  a  relatively  short  time.  The  late  1960's  and  early  1970's  provided 
the  first  real  efforts  towards  bringing  system  discipline  to  the 
acquisition  process  with  the  establishment  of  the  Defense  Systems 
Acquisition  Review  Council  (DSARC) ,  the  Blue  Ribbon  Defense  Panel,  and  the 
promulgation  of  Department  of  Defense  Instruction  (DODI)  5000.1. 

One  of  the  problems  that  has  plagued  the  acquisition  of  embedded 
computer  systems  is  that  the  management  technology  and  software 
engineering  skills  have  failed  to  keep  pace  with,  and  in  fact  have  never 
been  equal  to,  the  complexity  of  embedded  computer  systems.  These 
inadequacies  have  been  discussed  by  numerous  authors.  Tha  following  list 
of  references  are  but  a  few:  2/,  3/,  5/,  8/,  9/,  12/,  14/,  15/,  17/. 

This  dilemma  is  further  compounded  by  the  proliferation  of  management 
"guidance,"  and  in  the  ever-increasing  number  of  studies  aimed  at 
improving  the  acquisition  process.  There  is  in  fact  more  guidance  than 
can  be  successfully  applied  to  any  acquisition.  There  exists  a  problem  of 
determining  what  techniques  and  controls  should  be  applied  as  opposed  to 
what  must  be  applied. 

The  current  baseline  for  the  acquisition  of  major  systems  is  the  DSARC 
process.  4/  In  1976  in  response  to  recommendations  made  by  the  Commis¬ 
sion  on  Government  Procurement,  the  Office  of  Management  and  Budget  (OMB) 
issued  Circular  A-109,  Major  Systems  Acquisition.  In  response  to  this 
circular,  DoD  Instruction  5000.1  and  DoD  Instruction  5000.2  are  being 
revised  to  clarify  DoD  policy.  The  major  shift  in  the  emphasis  of  policy 
concerning  major  system  acquisitions  is  in  the  front  end  of  the  acquisi¬ 
tion  cycle.  Two  specific  areas  are  validation  of  the  mission  need  and  the 
increased  use  of  competition.  it  is  recognized  that  not  all  cases  of 
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acquisition  oc  modification  of  embedded  computer  systems  will  be 
designated  as  major  systems,  but,  as  will  be  discussed  in  Section  5,  it  is 
critical  that  additional  emphasis  be  placed  on  the  front-end  work  of  any 
ECS  if  true  improvement  in  the  quality  of  sizing  and  timing  estimates  is 
to  take  place. 

2 . 2  A  Baseline  System  Acquisition  Process. 

Figure  1  shows  the  acquisition  process  for  both  hardware  and  software, 
which  are  components  of  embedded  computer  systems.  This  process  is  an 
idealized  version  and  in  practice  is  rarely  followed  exactly  as  dis¬ 
played.  Paragraph  2.3  discusses  some  of  the  many  conditions  that  cause  a 
deviation  from  this  process,  and  the  associated  impact  on  sizing  and 
timing  estimates.  This  is  not  to  say  that  there  should  not  be  deviations 
from  the  acquisition  life-cycle,  in  fact  AFR  800-14  Volume  II  states  that 
a  program  may  skip  phases,  or  may  have  concurrent  activities  in  any  or  all 
phases. 

The  Conceptual  Phase  in  a  major  acquisition  begins  with  the  deter¬ 
mination  by  the  Secretary  of  Defense  that  a  mission  need  is  essential. 
This  determination  is  the  Milestone  0  decision,  program  initiation.  The 
primary  thrust  of  the  Conceptual  Phase  is  along  three  lines.  First,  there 
is  the  administrative  process  of  establishing  a  program  manager  with 

sufficient  authority  to  execute  the  second  major  conceptual  effort:  that 

of  investigating  alternative  design  concepts  and  recommending  the 
Validation  Phase  structure,  or  even  whether  or  not  to  proceed  with 

Validation  at  all.  The  third  thrust  of  the  Conceptual  Phase  is  the 

development  of  the  acquisition  strategy.  The  development  of  the  strategy 
provides  the  foundation  for  the  acquisition  plan  and  assists  the  program 
manager  in  defining,  as  is  presently  known,  the  path  that  a  program  will 
take.  The  ultimate  direction  of  a  program  should  depend  upon  the  results 
obtained  in  the  Conceptual  and  Validations  Phases. 


During  this  phase  all  sources  available  for  the  development  of 
competing  concepts  should  be  considered;  industry,  non-profit  organiza¬ 
tions,  government  laboratories  and  educational  institutions.  Parallel 
short-term  study  contracts,  coupled  with  experimental  validation  of  high 
risk  or  innovative  concepts  should  be  the  rule.  With  the  increased 
emphasis  on  competition  and  alternative  approaches,  the  primary  product  of 
the  Conceptual  Phase,  the  system/system  segment  specification  (Type  A), 
becomes  even  more  critical. 

Following  the  Conceptual  Phase  in  a  major  acquisition  is  the  Vali¬ 
dation  Phase.  This  phase  is  designed  to  provide  reduction  in  risk  to  the 
point  where  the  next  phase  may  be  commenced,  to  select  the  best  alterna¬ 
tive  system,  and  to  continue  to  develop  the  acquisition  strategy  by 
soliciting  and  evaluating  proposals.  This  phase,  for  both  major  and  minor 
acquisitions,  is  highly  critical  and  as  is  discussed  later,  is  often 
either  disregarded  or  combined  with  the  Conceptual  Phase,  with  Full-Scale 
Development  following  the  combined  Conceptual/Validation  Phase.  The 
product  of  the  Validation  Phase  is  primarily  the  B  Specification  (Part  I 
Development).  During  this  phase  the  System  Requirements  Review  (SRR)  and 
System  Design  Review  (SDR)  are  conducted,  and  the  Allocated  Software 
Baseline  and  Functional  Hardware  Baselines  are  established. 

The  final  phase  in  the  acquisition  process  is  the  Full-Scale  Develop¬ 
ment  Phase,  where  the  critical  software  construction  efforts  occur,  the 
Allocated  Hardware  Baseline  is  established,  followed  by  the  Hardware  and 
Software  Product  Baselines.  The  Full-Scale  Development  Phase  is  started 
by  the  DSARC  II  decision  and  involves  a  major  commitment  of  funds  for  the 
complete  engineering  development  of  the  system,  the  procurement  of  long- 
lead  items,  and  the  many  tasks  associated  with  preparing  for  production. 

2.3  ECS  Software  and  ECS  Acquisition  Fluidity. 

The  process  for  the  development  of  a  system  containing  an  embedded 
computer  system  is  by  its  very  nature  one  that  is  full  of  tradeoffs, 
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redirections,  and  competing  goals,  and  is  subject  to  a  host  of  external 
pressures.  The  actual  acquisition  process  for  a  major  system  is  rarely  as 
outlined  in  OMB  Circular  A-109.  In  some  cases,  an  upgrade  to  an  existing 
system  may  result  in  more  technical  problems  than  those  encountered  in  the 
acquisition  of  a  new  system.  There  are  a  few  basic  combinations  of  the 
acquisition  or  modification  of  a  system  with  embedded  computer  resources, 
but  there  will  always  exist  the  possibility  of  a  new  twist,  another 
combination,  that  will  result  in  a  deviation  from  the  acquisition  olan,  no 
matter  how  well  constructed.  For  example  a  new  threat  may  require  the 
backfitting  of  a  capability  that  was  never  considered,  or  one  that  has 
consciously  been  excluded,  which  is  even  more  difficult.  Some  of  the 
possible  conditions  that  may  exist  for  a  new  system  acquisition  are: 


•  Concurrent  hardware  and  software  development-  is 
required. 


• 

Adequate  hardware  in 
development  is  required. 

available 

and 

on  ly 

software 

• 

Adequate  software  is 
development  is  required. 

available 

and 

only 

hardware 

•  Only  an  upgrade  of  existing  hardware  and  software  is 
required. 


Once  the  baseline  structure  is  established,  there  exists  the  problem 
of  phasing  the  activities  within  the  process  The  acquisition  process 
taken  from  a  macro  approach  should  be  orderly  and  manageable,  but  rarely 
is.  Therefore,  the  procedures  and  techniques  developed  herein  are  based 
on  two  major  premises: 


1.  That  certain  factors  must  be  known  at  certain  points  in 
the  acquisition  process  to  support  reasonable  sizing 
and  timing  estimates. 

2.  That  in  the  event  these  factors  are  not  known,  steps 
may  be  taken  to: 

a)  estimate  the  missing  factors, 

b)  provide  a  measure  of  risk  in  not  having  the  actual 
data,  and 

c)  provide  for  the  accelerated  acquisition  of  the 
missing  data. 


9 


It  is  not  acceptable  to  assume  that  just  because  a  System/System 
Segment  Specification  is  delivered,  that  the  delivery  constitutes  an 
appropriate  increase  in  the  system  definition.  The  development  of  sizing 
and  timing  estimates  must  be  viewed  as  a  continual  refinement  of  initial 
estimates,  progressing  towards  a  completely  defined,  developed,  and 
operating  system,  to  the  maximum  extent  possible  in  the  system's  acqui¬ 
sition  life-cycle. 
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3.  ESTABLISHMENT  OF  THE  ECS  REQUIREMENTS 


The  establishment  of  the  ECS  requirements  is  the  single-most  important  key 
to  the  successful  development  of  a  c3  system.  10/ 

This  statement  may  be  considered  obvious  by  some,  and  absolutely  wrong 
by  others,  but  the  fact  that  there  is  disagreement  on  the  major  cause  of 
problems  associated  with  the  development  of  C3  systems  should  be  a 
matter  of  concern.  There  is  a  definite  gap  in  the  ability  of  system 
developers  to  adequately  translate  total  system  requirements  into  ECS 
requirements.  Often  the  information  required  to  adequately  size  a  system 
will  not  be  known  until  the  system  is  developed.  10/ 

In  addressing  the  ECS  requirements  of  a  c3  system,  there  are  many 
objectives  that  must  be  evaluated.  Based  on  Martin  13/,  a  listing  of 
factors  that  should  be  considered  prior  to  accepting  any  system  design  is 
presented  in  Table  1.  It  is  unlikely  that  all  the  items  will  be  fully 
answered  until  the  system  is  actually  complete,  but  the  items  are 
primarily  those  that  must  be  considered,  or  at  least  with  which  project 
personnel  must  be  most  familiar.  If  specific  data  can  not  be  obtained, 
then  there  must  be  a  reliable  method  to  alert  the  key  program  office 
personnel  of  the  potential  impact  of  not  having  the  data,  and  also  a 
method  that  will  schedule  the  time  when  the  data  will  become  available. 
In  a  competitive  environment  there  will  be  different  detailed  approaches 
to  solve  the  system  problem,  but  to  be  able  to  indicate  the  level  of 
detail  available  from  competing  offerors,  a  common  format  will  be  of 
assistance. 

Table  1  is  by  no  means  exhaustive,  nor  is  it  possible  to  have  all  the 
data  prior  to  committing  to  a  system  design.  What  is  important  to  note  is 
that  sizing  and  timing  estimates  are  a  part  of  the  system  design  and 
cannot  be  developed  with  any  degree  of  accuracy  outside  the  knowledge 
base,  such  as  the  factors  listed  in  Table  1,  that  bounds  workload, 
equipment,  and  total  project  resources.  The  more  accurate  the  data 


TABLE  1.  FACTORS  TO  BE  CONSIDERED  PRIOR  TO  ACCEPTANCE  OF  A  SYSTEM  DESIGN 


IMPACT  ON 

ESTIMATION  OF 

FACTORS 

SIZING 

TIMING 

REQUIREMENTS  DEFINITION 

H 

H 

Current  system  configuration  (if  applicable) 

H 

H 

Degree  of  real-time  operation 

M 

H 

Functional  capabilities 

L 

L 

Number  of  operators 

M 

M 

Reliability/Maintainability 

L 

L 

System  locations 

M 

H 

System  throughput 

M 

H 

System  turnaround  time 

SYSTEM  WORKFLOW 

M 

H 

Anticipated  average  CPU  utilization 

M 

M 

Anticipated  changes  in  workload 

M 

M 

Anticipated  size  of  outputs 

L 

M 

Average  workload 

M 

M 

Background  jobs 

M 

H 

Critical  workload 

H 

M 

Definition  of  permissible  degradation 

M 

H 

Degree  of  multiprogramming 

H 

M 

Hardware  and  software  utilization,  by 

job 

M 

H 

Job  arrival  distributions 

M 

H 

Job  arrival  rates 

L 

L 

Job  class  definition 

L 

M 

Job  priorities 

M 

H 

Job  service  times 

L 

M 

Maximum  I/O  rates  anticipated 

H 

M 

Number  of  output  reports 

L 

L 

Override  capabilities 

M 

H 

Peak  workload 

L 

M 

Permissible  off-line  jobs 

M 

H 

Probability  of  meeting  critical  period 

workloads 

H 

H 

Required  reserve  processing  capacity 

M 

M 

Scenario  definition 

L 

L 

Security  requirements,  by  job 

L 

H 

System  timing  diagrams 

L 

M 

Testable  scenarios  and  jobs 

HARDWARE  REQUIREMENTS 

M 

M 

Buffers 

H 

H 

Candidate  computer  configurations 

H 

H 

cnaracter istics  or  cpu 

LEGEND 

N 

H 

Communication  line  speeds 

H»High  Impact 

M 

M 

Hardware  functions  in  firmware 

M>Hediuo  Iapact 

N 

L 

Hardware  monitors 

L>Low  Impact 
N«N*qligible  Impact 

_ 

12 


TABLE  L.  FACTORS  TO  BE  CONSIDERED  PRIOR  TO  ACCEPTANCE  OF  A  SYSTEM  DESIGN 

(continued) 


IMPACT  ON 

ESTIMATION  OF 

FACTORS  | 

SIZING 

TIMING 

| 

HARDWARE  REQUIREMENTS  (continued) 

M 

M 

I/O  handlers 

L 

H 

I/O  speeds 

L 

M 

Input  device  configurations 

H 

M 

Memory  configuration 

H 

M 

Memory  expansion  capability 

L 

M 

Output  device  configurations 

1 

SOFTWARE  REQUIREMENTS 

H 

Application  programs 

H 

Compilers 

M 

Complexity 

N 

Development  facilities  available 

N 

Error  Correction 

L 

Module  execution  times 

!  H 

Operating  system  architecture 

N 

Program  flow  chart 

L 

Programming  languages 

L 

M 

Reliability 

H 

■ 

Size  estimate  of  developed  software 

M 

Software  monitors 

M 

Utilities 

FILE  STRUCTURE 

M 

Accounting  files 

L 

M 

Addressing  methods 

N 

N 

Data  security  classification 

L 

File  protection 

M 

File  structures 

Historical  data  requirements 

M 

Off-line  storage 

H 

On-line  storage 

L 

Predicted  file  growth 

M 

Type  or  data  storage 

LEGEND 

M 

Temporary  job  files 

H»High  Impact 

Test  file  data 

M»Mediur«  Impact 

L»Low  Impact 

| 

N*Nea’  iqible  Impact 
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obtained  to  the  above  factors,  the  better  will  be  the  estimate  of  system 
sizing  and  timing  parameters.  Table  1  may  be  used  as  an  initial  checklist 
to  assess  the  knowledge  available  for  input  to  the  estimation  process. 


3 . 1  Developing  and  Monitoring  ECS  Requirements. 

The  development  of  ECS  requirements  is  an  extremely  difficult  process 
that  often  becomes  even  more  difficult  as  the  acquisition  life-cycle 
progresses.  Several  factors  contribute  to  this  increasing  difficulty: 


•  Early  requirements  ate  viewed  as  "estimates  only"  which 
will  be  assumed  to  change  anyway. 

•  The  ECS  requirement  is  not  the  system  being  developed, 
but  only  a  portion  of  the  electronic  C3  system  that 
is  being  developed  to  meet  a  requirement. 

•  The  further  into  the  acquisition  life-cycle  a  program 
progresses,  the  shorter  the  lead  time  available  to 
modify  earlier  decisions,  which  may  have  been  incorrect. 

•  There  still  exists  a  feeling  that  in  the  event  of  an 
incorrect  hardware  decision,  problems  may  be  corrected 
in  software. 

•  Requirements  do  change,  even  well  into  Full-Scale 
Development,  and  it  becomes  harder  to  reestablish  the 
desired  balance  among  the  elements  of  the  ECS  and 
available  resources. 

The  degree  of  uncertainty  of  the  ECS  requirements  should  ideally 
decrease  with  increased  baseline  definitions.  However,  in  many  cases,  and 
all  too  often,  initial  estimates  of  the  ECS  requirements  fail  to  consider 
that  there  does  exist  a  degree  of  uncertainty  in  what  will  be  the  final 
configuration.  It  has  been  shown  by  Herd  9/  that  initial  estimates  on 
program  size  may  be  in  error  by  200%  in  the  conceptual  stage.  Early 
estimates  tend  to  be  viewed  as  more  accurate  than  they  are.  Later  in  the 
process  when  all  the  indicators  suggest  that  there  may  be  a  problem  with 
the  software  size,  people  tend  to  deny  the  existence  of  the  problem. 


One  of  the  reasons  that  ECS  sizing  and  timing  problems  are  recognized 
too  late  is  that  there  is  no  uniform  application  of  a  methodology  to 
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establish,  budget,  and  track  compliance  with  the  initial  and  revised 

estimates.  There  are  several  Data  Item  Descriptions  (DIDs)  that  provide 
for  the  submission  of  such  estimates,  but  these  are  not  routinely  applied 
throughout  ESD  acquisitions.  The  critical  factor  is  not  so  much  that  a 

particular  DID  is  used  or  not  used,  but  that  the  information  is  not 

obtained.  Further  analysis  of  specific  DIDs  is  contained  in  Section  6.  In 
addition  to  the  submission  of  estimates  by  contractors.  Program  Office 
personnel  and  Technical  Representatives  should  conduct  periodic  on-site 
reviews  of  the  contractor's  backup  data  for  their  sizing  and  timing 

estimates.  These  on-site  reviews  should  be  coordinated  with  regularly 
scheduled  program  reviews  that  should  occur  at  least  every  three  months. 


3 . 2  Requirements  Considerations  and  Problem  Areas. 

The  requirements  problems  associated  with  embedded  computer  systems 
within  systems  are  complicated  by  several  factors: 

•  C3  systems  are  inherently  complex  because  of  the 

random  nature  of  workloads  and  requirements. 

•  C3  systems  are  usually  "one  of  a  kind,"  and  therefore 

there  exists  no  large  data  base  from  which  to  extra¬ 
polate  data  to  be  used  in  estimates  for  future  systems. 

•  C3  functions  for  a  given  system  are  subject  to  large 

changes  because  of  the  changing  nature  of  the  threats 
with  which  the  military  must  deal. 

•  There  exists  no  catalog  of  approved  applications  soft¬ 
ware  designed  for  a  function.  In  addition,  there  are 

no  standards  for  defining  functions.  This  is  analogous 
to  designing  new  integrates  circuits  for  each  new 
application. 

•  Recent  trends  in  the  procurement  of  major  weapon 

systems  dictate  that  many  technical  approaches  should 
be  evaluated  in  response  to  a  government  requirement. 

The  process  permits  direct  comparisons  of  cost  and 
schedules  of  proposed  approaches,  but  makes  the 
comp ac i son  of  technical  system  parameters  difficult. 

If  competing  contractors  were  bidding  to  detailed 
specifications,  unrealistic  parameters  of  ECS  sizing 
and  timing  would  be  easier  to  identify,  in  that  the 
extremes  of  a  data  group  would  be  apparent. 


•  Generally  it  is  the  large  major  contractors  who  are 
involved  in  C3  system  acquisition,  and  this  in  itself 
further  reduces  the  number  of  approaches  that  may  be 
compared . 

•  The  length  of  time  to  acquire  major  systems  further 
complicates  the  problem  of  refining  estimates,  because 
long  periods  of  time  pass  between  estimates  and 
demonstrated  achievements. 

•  Since  much  of  the  hardware  for  C3  systems  may  be  in 
existence,  frequently  the  time  allotted  for  Request  For 
Proposal  (RFP)  preparation  is  too  short. 

These  are  some  of  the  factors  that  result  in  less  than  adequate 
requirements  definition.  Failure  to  adequately  define  requirements  will 
haunt  a  program  to  the  bitter  end. 


3.2.1  Conceptual  Phase.  The  Conceptual  Phase  in  the  system  acquisition 
life-cycle  is  the  time  during  which  alternative  concepts  are  developed  and 
evaluated  to  determine  which  will  De  responsive  to  established  mission 
requirements.  With  respect  to  computer  resources,  AFR  800-14  Volume  II, 
Acquisition  and  Support  Procedures  for  Computer  Resources  in  Systems, 
states: 

...(the  Conceptual  Phase)  is  the  initial  planning 
period  where  the  technical  military,  and  economic  bases 
are  established  through  comprehensive  studies,  experi¬ 
mental  development  and  concept  evaluation.  ...the 
major  definitive  document  resulting  from  this  phase  is 
the  initial  system  specification  which  documents  total 
system  performance  requirements. 

It  is  acknowledged  that  the  Conceptual  Phase  is  a  highly  iterative 
process,  and  therein  lies  one  of  the  major  problems  associated  with  sizing 
and  timing  of  ECS  systems  in  the  early  stages.  The  problem  is  that 
estimates  of  software  size  are  often  used  as  the  input  for  estimating 
initial  resources  required  in  justification  for  transition  into  the 
Validation  Phase,  and  yet,  software  size  is  also  an  output  which  must  form 
the  basis  for  the  operational  system  in  meeting  the  requirement.  There 
are  numerous  competing  objectives  that  must  be  treated  separately  as  well 
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as  in  concert.  Some  critical  factors  to  recognize  are  the  purpose  of  the 
sizing  and  timing  estimate,  the  degree  of  accuracy  possible,  and  above  all 
the  assumptions  that  go  into  the  estimate,  as  discussed  in  Section  5. 

The  output  of  the  Conceptual  Phase,  the  System  Specification, 
developed  in  accordance  with  MIL-STD  483  (as  modified  by  ESD  guidance),  is 
critical  to  successful  program  completion.  The  degree  of  detail  in  the 
system  specification  should  not  preclude  the  possibility  of  alternate 
solutions  ir.  ^aosequent  phases,  and  yet  to  provide  even  reasonable 
estimates  of  ECS  development  costs  for  the  Program  Decision  (entry  into 
the  Validation  Phase)  there  must  be  detailed  definition  of  the  software, 
even  to  the  level  of  definition  of  file  size  and  structure,  data  output, 
and  the  relationships  among  other  programs.  The  alternative  is  to  use 
macro  estimating  techniques,  recognizing  that  large  uncertainties  are 
involved,  and  be  prepared  to  accommodate  the  worst-case  situation. 

3.2.2  Validation  Phase.  In  keeping  with  the  increased  emphasis  on 

retaining  competition  in  the  early  acquisition  phases  and  the  exploration 
of  several  alternative  approaches,  the  Validation  Phase  has  taken  on 
increased  importance.  Yet  historically,  this  phase  has  often  been 
bypassed.  Whether  or  not  the  Validation  Phase  is  omitted  and  Full-Scale 
Development  follows  the  Conceptual  Phase,  the  fact  remains,  the  degree  of 
definition  provided  by  validation  will  still  have  to  be  done!  There  is  no 
way  around  the  requirement  to  progress  from  an  ill-defined  system  to  a 

working  system  except  by  increasing  one's  knowledge  of  the  system.  Two 

critical  reviews  that  should  be  conducted  during  the  Validation  Phase  are 
the  System  Requirements  Review  (SRR)  and  the  System  Design  Review  (SDR) . 
MIL-STD- 152 1A  provides  guidance  on  the  conduct  and  requirements  of  the  SRR 
and  SDR. 

3.2.3  Full-Scale  Development  Phase.  The  Full  Scale  Development  Phase 

(FSD) ,  including  limited  production,  is  initiated  when  the  need  of  the 

system  has  been  reaffirmed,  and  the  work  of  the  preceding  stages  indicates 
the  soundness  of  the  selected  approach.  The  software  Allocated  Baseline 


is  to  be  complete  at  this  point,  and  the  hardware  Allocated  Baseline 
shortly  after  the  start  of  FSD .  The  primary  purpose  of  the  FSD  is  the 
design,  fabrication  and  testing  of  the  system  under  development.  The 
output  of  this  phase  is  a  system,  closely  resembling  the  production 
system,  that  has  been  tested  and  documented  sufficiently  to  permit  a 
production  go  ahead. 

The  FSD  phase  contains  the  most  labor-intensive  effort  of  the 
acquisition:  actually  designing,  coding,  testing  and  integrating  the 
software.  During  this  phase  is  the  specifications  transition  from  the 
development  to  the  product  specification.  The  major  reviews  in  this  phase 
are  the  Preliminary  Design  Review  and  the  Critical  Design  Review. 

In  a  large,  one-of-a-kind  system,  there  may  not  be  a  "production"  of  a 
system,  but  only  a  militarization  of  parts  of  the  system.  From  the 
viewpoint  of  sizing  and  timing,  two  very  critical  activities  take  shape. 
First,  monitoring  the  ECS  development  and  second,  conducting  the  test  and 
evaluation  activities.  When  the  test  plans  and  procedures  are  being 
developed  it  is  critical  to  retain  the  trail  from  requirements  to 
workload,  to  test  conditions,  and  finally  to  test  procedures. 


4.  OVERVIEW  OF  DEVELOPMENT  OF  ECS  SIZING  AND  TIMING  ESTIMATES 


There  exists  no  agreed-upon  definition  of  sizing  and  timing  esti¬ 
mates.  The  generation  of  estimates  of  parameters  of  ECS  sizing  or  timing 
is  a  process,  with  the  result  being  a  deterministic  figure  that  is  in 
appropriate  units  corresponding  to  the  parameter  that  was  chosen  to  be 
measured.  System  sizing  parameters  could  be  any  or  all  of  the  following, 
or  more: 


•  number  of  lines  of  source  or  object  code  required  for 

the  operating  system  software, 

•  number  of  lines  of  source  or  object  code  required  for 

the  application  software, 

•  number  of  software  functional  modules  determined  to  be 
included  in  the  above, 

•  number  of  lines  of  object  code  required  to  be  resident 
in  the  main  memory  at  any  one  given  time, 

•  amount  of  data  and  programs  that  can  be  maintained 

offline, 

«  number  and  types  of  Input/Output  (I/O)  terminals  that 

are  required  for  proper  operation  of  the  system, 

•  capacity  required  for  the  main  memory  in  order  for  the 
system  to  perform  within  the  time  constraints  of  the 
operating  environment,  and 

•  capacity  of  the  mass  memory  that  is  required  for  stor¬ 
age  of  all  necessary  data  and  programs. 


This  points  out  that  when  discussing  sizing  of  an  ECS  system,  one  must 
determine  what  segment  of  the  ECS  is  specifically  being  addressed  in  order 
to  even  begin  to  discuss  the  subject. 


System  timing  is  an  equally  complex  process  to  bound.  Timing  may  have 
different  meanings,  depending  upon  the  circumstances  and  conditions  of  the 
discussion.  Examples  of  timing  parameters  are: 
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•  elapsed  time  occurring  between  the  end  of  a  query  input 
at  a  terminal  and  the  start  of  the  response  output, 

•  time  required  for  the  execution  of  a  software  module's 
processing  action, 

•  throughput  time  required  to  complete  the  necessary 
processing  of  an  estimated  peak  workload, 

•  queuing  times  estimated  for  each  I/O  device,  memory  or 
CPU, 

•  time  required  to  update  various  data  bases, 

•  utilization  time  of  CPU  or  other  components  that  is 
determined  for  a  specific  workload,  and 

•  time  required  to  transfer  data  through  communication 
lines  or  networks. 

Once  again  the  specific  time  condition  to  be  estimated  must  be 
defined.  In  addition  to  defining  the  particular  sizing  or  timing 
estimate,  the  purpose  of  the  estimate  may  dictate  what  technique  or 
combination  of  techniques  might  be  most  applicable  (see  Section  4.3). 


4. 1  Sizing  and  Timing  Approach. 

The  primary  parameter  of  any  sizing  and  timing  study  is  the  purpose 
for  which  the  study  is  being  conducted.  This  initial  entry  point  in  the 
process  was  chosen  for  the  following  four  reasons: 


•  The  functional  purpose  of  the  study  is  not  dictated  by 
the  phase  of  the  acquisition  Jife-cycle,  rather  by  the 
procedures  and  data  available. 

•  The  acquisition  life-cycle  of  c3  systems  with 
embedded  computer  systems  will  normally  vary  with  each 
system,  therefore  attempting  to  dictate  a  particular 
study  approach  in  a  particular  acquisition  life-cycle 
phase  would  severely  limit  utility. 

•  The  quality  of  the  system  definition  and  the  quality  of 
documentation  are  subject  to  wide  variations. 
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•  Since  the  procedures  must  be  applicable  to  all  ESD 
acquisitions,  they  must  permit  accommodation  of  a  wide 
range  of  system  definitions,  for  example,  in  some  cases 
hardware  may  be  dictated,  in  others  not. 

Coupled  with  the  purpose  of  the  study  and  candidate  procedures,  is  the 
overall  logic  behind  any  sizing  and  timing  study.  Here  again,  the  require¬ 
ment  that  the  procedures  be  applicable  to  all  acquisition  life-cycle 
phases,  and  to  all  ESD  c3  acquisitions,  dictates  that  a  sound,  adaptable 
study  logic  be  developed,  based  on  engineering  disciplines.  The  logic 
diagram  for  the  sizing  and  timing  studies  proposed  herein  is  shown  in 
Figure  2.  This  logic  is  similar  to  the  work  conducted  by  Gilbert,  et  al., 
of  the  Federal  Computer  Performance  Evaluation  and  Simulation  Center 
(FEDSIM) .  U  Additional  emphasis  has  been  placed  in  two  areas:  first,  in 
an  expansion  of  the  effort  to  permit  utility  in  the  early  phases  of  the 
acquisition  life-cycle;  and  second,  an  increased  emphasis  on  measurement 
of  the  uncertainty  associated  with  the  results  of  the  study. 

The  concept  of  integrating  the  sizing  and  timing  estimates  within  a 
single  study  framework  is  based  upon  the  common  denominator  of  WORKLOAD. 
The  workload  of  the  ECS  drives  the  system  configuration  and  results  in  a 
given  system  performance.  The  workload  is  the  transition  variable  between 
system  requirements  and  system  performance,  and  yet  is  extremely  difficult 
to  quantify  early  in  an  acquisition.  Figure  3  illustrates  the  relation¬ 
ships  between  requirements,  workload,  the  ECS,  and  the  C3  system.  The 
ECS  must  respond  to  the  workload  imposed  by  the  C3  system  requirements. 
If  the  workload  does  not  support  the  requirements,  the  ECS  can  not  support 
the  C3  system  and  will  result  in  inadequate  outputs  or  poor  system 
performance. 
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Figure  2-  Overview  of  Siting  and  Timing 


! 


Figure  3.  Requirements  and  Workload  Relationship 
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The  workload  of  an  embedded  computer  system  as  used  in  this  report  is 
defined  as  the  total  demand  placed  upon  the  embedded  computer  system  by 
the  user  in  a  specified  period  of  time. 

Much  has  been  written  about  the  lack  of  requirements  definition  being 
a  major  contributor  to  the  development  of  systems  that  exceed  cost  targets 
and  fail  to  meet  performance  requirements.  This  is  undoubtably  true? 
however,  another  large  contributor  has  been  the  lack  of  a  workload 
definition  as  the  bridge  between  requirements  and  system  design.  For 
example,  a  detailed  breakdown  of  application  software,  on  a  module- 
by-module  basis,  may  provide  an  accurate  measure  of  ECS  storage 
requirements,  but  would  contribute  little  by  itself  to  the  problem  of 
system  timing.  In  addition,  even  the  execution  time  of  each  compiled 
module  would  be  of  no  benefit  unless  the  sequence  of  execution  for 
specific  workloads  was  provided.  Accordingly,  one  of  the  purposes  of  this 
Handbook  is  to  integrate  the  conduct  of  sizing  and  timing  studies  with  the 
concept  of  workload  definition. 

AFR  800-14  volume  II  addresses  several  components  of  workloads,  such 
as  "minimum  iteration  rates  for  various  functional  processing,"  (para  3-4 
f (4) ) ,  "functions  are  arranged  in  their  logical  sequence  so  that  any 
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specified  operational  usage  of  the  system  can  be  traced  in  an  end-to-end 
or  in  a  closed-loop  path,"  (para  4-5a(l)).  In  addition,  paragraphs  4-9c 
and  d  require  analysis  of  critical  timing  requirements  and  run-times  at 

the  Preliminary  Design  Review  (PDR) ,  and  review  of  processing  time  and 
memory  estimates  at  the  Critical  Design  Review  (CDR) .  Both  of  these 
requirements  have  workload  implications.  Also  paragraph  5-6a  discusses 
computer  program  Verification  and  Validation,  wherein  it  states  that 
during  the  analysis  phase,  "a  timing  and  sizing  study  should  be  conducted 
to  insure  that  the  proposed  computer  system  is  adequate." 

In  order  to  complete  the  overview  of  ECS  sizing  and  timing  it  is 

necessary  to  provide  guidelines  as  to  what  the  output  of  the  studies  will 
be,  in  well  defined  units.  It  would  not  be  possible  to  provide  a  list  of 

all  sizing  and  timing  output  measures.  The  primary  reason  for  this  is 

that  the  output  is  determined  or  dictated  by  the  objective  of  the  study. 
For  example,  the  primary  measure  of  size  could  be  words  of  main  memory,  or 
number  of  terminals,  or  even  the  character  capacity  of  a  Cathode  Ray  Tube 
(CRT).  Therefore,  as  used  in  this  Handbook,  size  is  defined  as: 

The  size  of  an  embedded  computer  system  has  two  components; 

1)  physical  size:  in  terms  of  the  number  and  types  of 
physical  hardware  units  and  the  number  of  unique  software 
entities  including  data  and  their  corresponding  number  of 
source  statements,  object  words,  or  required  storage  space 
as  appropriate;  and  2)  capacity:  in  terms  of  storage  or 
processing  capability  of  hardware  that  is  not  dependent 
upon  software. 

The  precise  boundaries  of  an  embedded  computer  system  will  vary 
depending  on  the  specific  system.  A  conceptual  c3  ECS  configuration  is 
provided  in  Figure  4.  The  number  of  components  and  interfaces  will  vary 
from  system  to  system. 

The  timing  of  an  ECS  is  also  subject  to  wide  variation  and  is  depen¬ 
dent  upon  whatever  the  individual  conducting  the  study  has  established  as 
the  objective.  The  following  definition  of  timing  of  embedded  computer 
systems  forms  the  basis  for  the  remainder  of  this  Handbook: 
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COMMAND,  CONTROL  AND  COMMUNICATIONS  (CJ)  SYSTEM 


Conceptual  C  Embedded  Computer  System  (ECS) 


The  timing  of  an  embedded  computer  system  has  two 
components;  1)  workload  independent  parameters  such  as 
memory  access  time,  cycle  time  and  printer  output  rate,  and 
2)  workload  dependent  measures  such  as  THROUGHPUT,  the  work 
completed  per  unit  time  for  a  given  workload,  RESPONSE 
TIME,  the  elapsed  time  between  the  finish  of  a  request  and 
the  start  of  the  output  for  a  given  workload,  and  UTILIZA¬ 
TION,  the  ratio  of  the  time  spent  by  a  device  performing 
work  during  a  specified  interval  to  the  total  interval,  at 
a  given  workload. 


There  are  many  other  derived  parameters  that  may  be  considered 
measures  of  system  performance.  However,  the  concept  of  throughput,  res¬ 
ponse  time,  and  utilization  answers  three  basic  questions  regarding  an 
entire  ECS  or  a  specified  component: 

•  How  much  work  is  being  done? 

•  How  fast  is  it  being  done? 

•  How  much  more  work  can  the  system  do? 

Beizer  1/  has  stated  that  every  system  has  a  characteristic  curve  that 
relates  response  time  (delay)  to  throughput  similar  to  that  shown  in 
Figure  5. 


Figure  5.  Throughput  Vs.  Delay  Curve 
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It  must  be  recognized  that  throughput  and  response  time  (or  delay)  are 
extremely  complex  for  both  a  given  system  and  a  given  workload,  and  are 
composed  of  such  factors  as  job  mix,  job-arrival  rate,  job-processing 
time,  data  base,  code  construction,  and  more.  The  specific  components  of 
throughput  and  delay  are  discussed  further  in  Section  5. 

4.2  Study  Framework. 

4.2.1  Purpose  of  the  Sizing  and  Timing  Study.  Determining  the  purpose  of 
any  particular  sizing  and  timing  study  is  the  initial  entry  point  for  the 
start  of  the  sizing  and  timing  process.  Seven  potential  points,  during 
the  acquisition  life-cycle  of  ECS,  at  which  sizing  and  timing  studies 
would  be  beneficial,  are  discussed  in  sections  4.2.2  through  4.2.8.  It  is 
possible,  and  in  some  cases  desirable,  that  more  than  one  type  of  study  be 
conducted  at  the  same  time.  In  this  event,  the  multiple  studies  should  be 
defined  in  a  single  study  objective  as  discussed  in  Section  5.2.1  of  this 
Handbook. 

4.2.2  Initial  Resource  Estimates.  The  Conceptual  Phase  of  the  acquisi¬ 
tion  life-cycle  is  normally  when  initial  estimates  will  be  required. 
However,  cases  do  arise  when  a  new  system  may  be  required  to  interface 
with  an  existing  system  that  is  currently  in  Full-Scale  Development,  and 
the  sizing  and  timing  study  may  have  to  consider  the  existing  constraints 
of  the  more  mature  system,  but  this  is  not  the  norm.  A  primary  purpose 
for  conducting  an  early  sizing  and  timing  study  is  to  provide  a  basis  for 
derived  resource  requirements  needed  to  support  the  budget  process. 
Nearly  all  models  and  techniques  for  estimating  resources  for  the  develop¬ 
ment  of  software  use  a  measure  of  the  estimated  'size  of  the  software  as 
the  independent  variable  when  computing  manpower,  schedule,  computer 
usage,  and  hardware  size.  A  key  feature  of  any  estimates  derived  from 
initial  studies  is  that  they  should  be  used  to  measure  limiting  conditions 
of  the  overall  proposed  acquisition.  For  example,  one  could  question 
whether  the  estimated  code  could  reasonably  be  developed  within  the 
overall  program  schedule  and  manning.  An  estimate  of  system  sizing  and 


27 


timing  is  required  when  considering  the  initial  system  configuration.  The 
recent  emphasis  on  encouraging  alternate  system  concepts,  unconstrained  by 
hardware,  only  increases  the  need  for  a  more  formalized  approach  to 
conducting  sizing  and  timing  based  on  general  hardware  configurations. 
The  sizing  and  tiding  study  conducted  in  response  to  configuration 
questions  can  provide  limits  on  basic  system  parameters  such  as  maximum 
processing  speeds  available  from  a  given  central  processor,  or  required 
processing  speeds  for  given  program  size  and  workloads.  All  program 
resources  must  be  considered  in  relation  to  each  other  with  the  goal  of 
reducing  the  possibility  of  having  unattainable  goals  for  hardware, 
software,  time,  and  dollars. 

4.2.3  Comparison  of  Alternative  Concepts.  The  use  of  sizing  and  timing 

studies  in  support  of  the  evaluation  of  alternate  concepts  presents  an 
additional  set  of  problems.  Ii  iis  instance,  where  the  workload 
definition  may  differ  even  in  response  to  the  same  requirement,  the 
problem  of  comparing  like  performance  indicators  is  compounded.  The  major 
difficulty  is  weighing  the  utility  of  various  performance  parameters, 
specifically  throughput  and  response  time.  For  example,  one  might 
consider  the  marginal  utility  of  increasing  throughput  and  decreasing 

response  time.  The  other  factor  that  enters  the  equation  of  system 

comparison  is  cost.  Although  not  a  specific  output  of  a  sizing  and  timing 
study,  the  estimates  of  size,  as  described  above,  are  normally  used  as  the 
basis  of  cost  estimates.  There  are  additional  factors  that  are  inde¬ 
pendent  of  workload,  such  as  memory  access  speed,  that  will  enter  the 

compar ison. 

4.2.4  Preparation  of  Procurement  Documents.  Sizing  and  timing  studies 
conducted  in  support  of  the  preparation  of  Requests  for  Proposals  (RFP) 
can  be  invaluable,  in  that  conducting  such  studies  will  assist  in  the 
process  of  translating  system  requirements  into  workloads.  This  in  turn 
will  provide  a  better  basis  for  structuring  Statements  of  Work  (SOW)  and 
the  development  of  the  evaluation  of  proposed  systems.  There  exist 
competing  objectives  for  specifying  enough  mandatory  requirements,  while 
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not  biasing  the  RFP  in  favor  of  a  particular  system.  In  some  cases, 
however,  if  the  acquisition  is  a  modification  to  an  existing  system,  the 
preexisting  constraints  may  severely  limit  the  options  cf  responding 
contractors. 

4.2.5  Proposal  Evaluation.  The  evaluation  of  competitive  and  sole-source 
proposals  for  complex  systems  is  an  extremely  difficult  and  time-consuming 
process.  The  Source  Selection  Evaluation  Board  (SSEB)  must  be  knowledge¬ 
able  in  areas  of  technical  documentation,  the  associated  cost  proposals, 
and  also  must  be  concerned  with  the  evaluation  criteria  and  contractual 
aspects  of  the  proposal  evaluation.  With  the  increasing  emphasis  on 
alternative  solutions,  the  direct  comparison  of  different  proposals  can  be 
difficult  at  best.  By  conducting  a  sizing  and  timing  study  from 
proposals,  based  on  the  common  definitions  of  sizing  and  timing  parameters 
as  defined  in  this  Handbook,  the  competing  capabilities  may  be  evaluated. 
This  is  based  on  the  requirement,  however,  that  in  the  RFP  guidance  for 
proposal  preparation,  the  basic  components  of  the  sizing  and  timing  input 
data  were  requested.  An  advantage  of  conducting  a  formal  sizing  and 
timing  study  during  proposal  evaluation  is  that  relative  comparisons  of 
throughput  and  response  time  may  be  made  from  the  study  workload  as 
opposed  to  discrete  numbers  which  may  or  may  not  have  been  derived  from 
identical  workload  assumptions. 

4.2.6  Development  Monitoring.  The  use  of  sizing  and  timing  studies 
during  development  of  an  ECS  can  support  other  management  reporting 
systems  such  as  Cost/Schedule  Control  Systems  Criteria  (C/SCSC) 
reporting.  C/SCSC  reports,  as  provided  by  the  Cost  Performance  Report 
(CPR)  for  major  systems,  are  primarily  concerned  with  the  financial  status 
of  the  program  and  with  the  concept  of  earned  value;  that  is,  what  costs 
were  budgeted  for  completed  work.  The  obvious  difficulty  is  that  it  is 
very  hard  to  convert  parameters  such  as  system  sizing  and  timing  into 
elements  of  cost.  One  possible  solution  is  to  plan  for  and  budget  for  the 
contractor  to  conduct  increasingly  refined  sizing  and  timing  studies  and 
use  the  quantifiable  reduction  in  sizing  and  timing  estimate  errors  as  the 


measurement  criteria.  All  too  often  initial  sizing  and  timing  estimates 
are  not  refined  until  the  start  of  integration.  The  fact  that  the 
execution  time  of  the  modules  are  known  is  no  guarantee  that  the 
higher-level  component  will  perform  to  satisfy  the  system  requirement. 
Sizing  and  timing  studies  conducted  by  or  reviewed  by  the  Program  Office 
(PO)  must  be  done  on  a  routine  basis.  The  longer  a  system  goes  without  a 
revision  or  update  of  a  sizing  and  timing  estimate,  the  more  suspect  the 
estimates  become  and  the  more  potentially  complex  future  corrections 
become,  both  in  terms  of  the  cost  of  corrections  and  decreased  reliability 
of  the  system.  The  key  is  scheduled  refinement  of  estimates  with 
measurable  verification. 

4.2.7  Changes  in  Requirements  or  Workload.  One  of  the  few  things  that 
may  safely  be  said  about  the  acquisition  of  C3  systems  is  that  there 
will  be  requirement  changes.  These  changes  may  take  three  forms: 
modifications,  deletions,  and  additions.  Modifications  to  requirements 
may  occur  as  a  result  of  several  factors,  such  as  a  clearer  definition  of 
what  the  requirements  really  were,  as  a  result  of  new  mission  needs,  or  as 
a  result  of  possibly  competing  objectives.  The  deletion  of  requirements 
is  most  likely  to  occur  as  a  result  of  three  factors:  the  cost  to  imple¬ 
ment  the  requirements,  the  need  to  remain  within  performance  constraints, 
and  the  need  to  meet  an  established  operational  target  date.  The  addition 
of  requirements  is  apt  to  take  place  early  in  the  acquisition  life-cycle 
as  a  result  of  user  desired  system  enhancements.  The  system  user  tends  to 
require  more  and  more  capability.  The  danger  here  is  that  the  cumulative 
effect  of  small,  early  changes  will  not  be  translated  into  additional 
workload.  Consequently,  when  the  system  is  tested,  its  overall 
performance  may  surprisingly  be  found  unacceptable. 

Sizing  and  timing  studies  should  be  conducted  to  evaluate  the  impact 
of  any  change.  The  degree  of  complexity  of  the  study  is  a  function  of  the 
detailed  knowledge  of  the  system  and  the  possible,  not  probable,  conse¬ 
quence  of  the  change.  Of  primary  concern  in  conducting  this  type  of  study 
is  the  definition  of  the  workload  change.  To  properly  evaluate  require- 
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ments  and  hence  workload  changes,  it  is  essential  that  the  baseline 
workload  be  tracked  throughout  the  entire  acquisition  life-cycle.  Section 
5  discusses  a  workload  tracking  and  update  system. 

4.2.8  System  Performance  Evaluation.  The  measurement  of  system  operation 
is  conducted  on  the  operational  system,  or  portions  thereof.  It  may  be 
used  to  exercise  the  system  against  a  benchmark  program.  The  benchmark 
program  may  or  may  not  describe  the  projected  system  workload;  however, 
the  benchmark  workload  has  a  specific  meaning  to  the  designer  who  may  be 
interested  in  comparing  specific  features  of  two  systems. 

Discrete  event  simulators  are  also  used  to  evaluate  a  system  that  may 
be  experiencing  unacceptable  response  time,  where  the  cause  of  the  bottle¬ 
neck  is  unclear.  The  development  of  large-scale  discrete  simulators  can 
be  very  costly,  but  may  reveal  system  choke  points,  which  may  then  be 

redesigned  within  the  simulation  and  reverified  for  performance 
improvement. 

4.2.9  Conclusion.  It  is  important  to  understand  the  relationships 

between  sizing  and  timing  estimates,  and  the  acquisition  life-cycle. 
Initially  system  timing  is  a  requirement,  and  the  process  of  making  timing 
estimates  is  for  purposes  of  (1)  insuring  that  the  timing  requirements 
(throughput,  response  time)  are  still  what  is  required,  and  (2)  that  the 
system  development  progresses  to  meet  those  requirements.  On  the  other 
hand,  system  sizing  is  an  estimate  of  what  will  be  required  to  meet  the 
functional  and  timing  requirements.  A  serious  budgetary  problem  may  occur 
in  the  acquisition  of  a  C3  system.  The  initial  ballpark  sizing 

estimates  provided  for  a  budget  may  be  assumed  to  be  specific  figures 

rather  than  estimates  by  financial  management  personnel.  Accordingly,  use 
of  estimate  figures  should  be  identified  and  explained  in  budget 
submissions. 

The  relationships  reverse  as  a  system  proceeds  through  development,  in 
that  the  size  becomes  fixed  as  software  is  developed  and  hardware  is 
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assembled.  Actual  timing  must  be  estimated  or  measured  to  verify  that  the 
requirement  is  met.  All  too  often  in  Full-Scale  Development  the  timing 
requirements  are  relaxed  because  the  size,  and  therefore  budgets,  cannot 
be  expanded.  The  way  to  attack  the  problem  is  to  continually  monitor  the 
relationships  among  sizing,  timing,  and  resource  parameters. 

4 . 3  Sizing  and  Timing  Techniques. 

Sizing  and  timing  techniques  are  presented  in  Figure  6.  As  indicated 
in  the  figure,  techniques  are  divided  into  three  classes:  EXTRAPOLATION 

techniques  that  may  be  used  in  the  early  phases  of  acquisition  when  the 
sizing  and  timing  parameters  of  the  proposed  ECS  are  difficult  to  define; 
REPRESENTATION  techniq  es  that  may  be  used  when  more  details  are  known 
about  the  sizing  and  timing  parameters;  and  MEASUREMENT  techniques  that 
may  be  used  in  the  latter  part  of  the  acquisition  to  determine  actual 
sizing  and  timing  parameters.  Considering  categories  of  techniques,  there 
is  no  clear  dist: iction  as  techniques  move  from  ANALYTICAL  to  SIMULATION. 
Many  authors  discuss  analytical  models  as  discrete  event  simulation 
models.  The  real  difference  is  the  assumptions  made  in  the  model 
construction,  particularly  in  the  assumption  of  the  distribution  assigned 
to  the  workload.  When  dealing  with  a  C3  ECS,  an  exact  mathematical 
analysis  of  the  ultimate  actual  workload  may  be  impossible,  and  even 
assumptions  of  job  arrival  times  may  pose  a  real  problem.  Techniques  are 
discussed  further  in  Sections  5.2.3,  5.3.4,  and  5.4.2,  and  Tab  A. 
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CLASS  Of  TlCHNlUUE 


Figure  6 


Sizing  and  Timing  Techniques 


5.  CONDUCTING  SIZING  AND  TIMING  STUDIES 


5 . 1  Study  Overview. 

The  basis  for  improved  estimates  of  ECS  sizing  and  timing  parameters 
lies  in  the  application  of  engineering  disciplines.  By  establishing  and 
using  standard  procedures  on  a  regular  basis,  estimators  (users)  will 
increase  their  understanding  of  sizing  and  timing  studies.  This 
understanding  should  improve  the  users'  ability  to  conduct  sizing  and 
timing  studies  and  to  objectively  analyze  the  studies  conducted  by  others. 

In  addition  to  using  these  procedures  on  a  regular  basis  when 
conducting  sizing  and  timing  studies,  one  should  insure  that  all  studies 
are  adequately  documented  and  retained  for  future  reference  in  Program 
Offices  and/or  ESD/TOI. 

The  sizing  and  timing  study  procedures  presented  in  this  handbook  are 
for  use  by  the  engineers  and  computer  scientists  of  the  Electronic  ./stems 
Division's  Computer  Systems  Engineering  Directorate  (ESD/TOI).  They,  in 
turn,  should  be  able  to  better  assist  other  ESD  personnel  in  conducting  or 
evaluating  sizing  and  timing  studies.  IN  THE  i.  .'ENT  ESD  PERSONNEL  NEED 
ASSISTANCE  IN  CONDUCTING  A  SIZING  AND  TIMING  STUDY  -  CONTACT  ESD/TOI. 


THE  APPLICATION  OF  'HE  PROCEDURES  DEPENDS  ON  HOW  MUCH  IS  KNOWN  ABOUT  A 
SYSTEM  RATHER  THAN  THE  SYSTEM'S  ACQUISITION  PHASE.  Availability  of  sizing 
and  timing  data  will  vary  depending  on  such  factors  as:  the  extent  of 
in-depth  data  contained  in  the  system's  definition  and  specifications  (for 
example  see  Table  1),  the  sizing  and  timing  data  -collection  requirements 
imposed  upon  the  contractor,  and  the  aggressiveness  of  the  Program  Office. 

Efforts  or  studies  to  estimate  the  sizing  and  timing  parameters  are 
divided  into  three  different  types  (see  Figure  7): 

•  Initial  Studies  are  usually  conducted  in  the  early 
stages  of  the  conceptual  system  definition  activity. 
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•  Update  Studies  should  be  conducted  at  critical  points 
throughout  the  system  acquisition  life-cycle.  These 
studies  should  be  done  when  any  significant  changes 
occur  that  might  impact  on  sizing  and  timing  parameters 
(also  see  Table  1  in  Section  2),  prior  to  major  program 
reviews  (also  see  Figure  1  in  Section  2),  or  any  other 
time  an  update  is  deemed  appropriate.  This  latter 
event  might  well  be  on  the  occasion  of  key  personnel 
changes  so  that  new  personnel  will  have  an 
understanding  of  how  prior  studies  were  conducted  and 
accordingly  develop  their  own  level  of  confidence  in 
the  developed  estimates. 

•  Developmental  Monitoring  consists  of  studies  that  are 
scheduled  regularly  during  the  acquisition  life-cycle. 
Though  monitoring  is  acknowledged  to  be  extremely 
important,  it  has  usually  not  been  performed  in  the 
past.  Lacking  encouragement,  it  has  been  all  too  easy 
to  go  for  months  and  even  years  without  updates,  or 
more  important,  verification  of  the  initial  sizing  and 
timing  estimates. 


The  procedures  specified  for  c-ach  type  of  sizing  and  timing  study  are 
discussed  in  detail,  step-by-step,  in  the  remainder  of  this  section.  The 
discussions  include  approaches  from  Gilbert,  et  al.  1_/ 

5 . 2  Conducting  Initial  Sizing  and  Timing  Studies. 

Procedures  for  conducting  initial  sizing  and  timing  studies  are 
presented  in  Figure  8  and  are  discussed  in  Sections  5.2.1  through  5.2.10. 


Figure  8.  Sizing  and  Timing  Initial  Study  Procedures 
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1.2.1  Set  Objectives.  The  objectives  of  a  study  should  identify  specific 
needs  such  as  initial  resource  estimates,  comparison  of  alternative 
concepts,  proposal  evaluation,  etc. 


When  structuring  the  objective  the  following  should  be  considered: 

•  Specify  the  parameter(s)  to  be  measured  in  precise 
terms,  such  as  determination  of  main  storage  given  80% 
utilization  at  the  workload  specified,  not  just  core 
required. 

•  Identify  the  resources  available  to  conduct  the  study, 
in  terms  of  personnel,  funds,  and  time. 

•  Provide  definitions  of  any  terms  to  be  used  that  are 
not  standard,  or  that  may  be  subject  to  interpretation. 

•  Specify  the  anticipated  degree  of  accuracy  of  the  final 
study  result(s) . 

•  Identify  other  organizations  and  individuals  who  will 
make  use  of  the  study  results. 

•  Do  not  specify  an  obviously  unrealistic  objective. 

•  Do  not  plan  to  consume  more  resources  than  would  be  the 
cost  penalty  of  not  doing  the  study  at  all. 


5.2.2  Define  System  Boundaries  and  Workload.  The  definition  of  the 

system  or  subsystem  workload  is  the  most  important  step  in  a  sizing  and 

timing  study.  It  may  not  always  be  possible  to  accurately  define  the 

workload,  especially  during  initial  sizing  and  timing  studies.  In  this 
case  a  notation  should  be  ma  to  this  effect  for  historical  purposes.  In 
bounding  the  system  and  define. .g  workload: 

•  Draw  or  identify  a  block  diagram  of  the  portion  of  the 
embedded  computer  system  under  evaluation. 

•  Specify  all  known  or  assumed  hardware  parameters.  If 
the  parameters  of  a  hardware  unit  are  the  object  of  the 
study,  identify  the  limits  of  the  parameter  set. 

•  Iden  '  fy  any  known  software  characteristics  that  fall 

withi..  the  ECS  boundary.  This  may  include  firmware 
(see  definition  in  Glossary)  considerations. 
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Identify  any  software  size  being  estimated  as 
accurately  as  possible.  It  may  be  no  more  than  simple 
application,  but  be  sure  to  define  what  is  meant.  For 
example,  if  the  software  is  for  controlling 
Input/Output  (I/O)  devices,  bound  it  by  defining  what 
devices  will  be  controlled.  Remember  to  include 
software  for  training  and  testing,  if  applicable. 

Define  as  many  workload  parameters  as  possible.  A 

complete  definition  will  include: 

Job  definition  -  A  job  is  an  action  by  the  ECS 
that  places  a  demand  on  system  resources.  This 
can  be  as  simple  as  reading  in  two  values  from 
disc,  adding  them  in  core  and  printing  one  number, 
or  it  can  be  as  complex  as  executing  a  subroutine 
with  thousands  of  executable  paths  in  a 
distributed  system. 

Job  sequencing  -  Identify  the  sequence,  or  the 
probability  distribution  of  job  calls. 

Job  run  time  -  Determine  the  time  required  for  a 
job  to  run.  This  can  be  a  discrete  value  for  a 
sequential  job,  or  expressed  as  a  probability  if, 
for  example,  there  exists  an  80%  probability  that 
a  job  will  execute  in  a  certain  manner,  then  the 
execution  time  of  the  path  may  be  calculated. 

System  resources  required  -  Identify  system 
components,  such  as  terminals,  discs,  core,  tapes, 
etc. 

Job  priority  -  Determine  the  priorities  of  jobs  in 
a  workload. 

Define  any  related  workloads,  such  as  that  of  a  system 
being  replaced. 

Identify  assumptions  -  This  is  important  in  that  often 
assumptions  lose  the  quality  of  an  unknown  and  assume  a 
measure  of  fact  if  they  are  not  verified  and  refined. 
Examples  of  assumptions  that  may  have  to  be  made  are: 

future  trends  in  performance  characteristics  of 
hardware  that  may  occur, 

any  workload  parameter,  such  as  job-arrival  time 
that  is  not  accurately  defintl,  and 


basic  system  configuration  that  is  not  accurately 
described. 


Once  the  system  boundaries  and  workloads  have  been  established,  care 
must  be  taken  to  insure  that  the  objectives  of  the  study  can  still  be 
met.  Even  if  there  are  numerous  assumptions,  as  long  as  they  are 
accounted  for,  the  study  will  have  meaning. 

5.2.3  Establish  Candidate  Techniques.  The  establishment  of  candidate 
techniques  (see  Figure  6)  is  a  function  of  how  much  is  known  about  the 
system  under  study,  the  study  objectives,  and  how  many  assumptions  may  be 
made  and  still  provide  a  useful  product.  It  must  be  kept  in  mind  that 
until  the  ECS  is  fully  developed  with  all  code  and  data  in  place,  the  size 
is  uncertain.  Also,  until  the  system  is  exercised  under  actual  operating 
conditions,  the  true  timing  is  uncertain.  In  conducting  a  sizing  and 
timing  study  there  may  be  portions  of  the  system  that  are  fully  developed 
ar  of f-the-s.'.elf ,  and  therefore,  some  of  the  direct-measurement  techniques 
may  apply.  In  the  same  study,  however,  the  logical  flows  of  certain  paths 
may  be  undefined  and  require  the  application  of  less  precise  techniques. 
The  point  to  bear  in  mind  is  that  all  classes  of  techniques,  i.e. 

extrapolation,  representation,  and  measurement ,  should  be  considered  in 

/ 

the  context  of  objectives  and  data  availability. 

When  selecting  the  candidate  technique ( s) : 

•  Refer  to  Tab  A  and  start  with  the  most  accurate 
technique  category,  such  as  monitors,  and  work 
backwards  towards  analogy. 

•  As  a  first  cut  at  selection,  be  more  concerned  whether 
the  required  data  can  be  obtained,  not  how  it  will  be 
obtained  or  how  much  it  will  cost.  This  will  result  in 
a  feasible  set  of  techniques,  but  not  necessarily  a 
viable  set. 

•  Understand  the  basic  assumption  required  for  the 

techniques  use.  For  example,  to  use  any  Markovian 

technique  11/,  it  must  be  assumed  that  if  a  sequence 


of  jobs  is  to  be  modeled,  the  next  step  in  the  model  is 
dependent  only  c  the  current  state,  and  not  upon  any 
previous  histor  In  some  cases  this  may  he  a 

reasonable  ass'  cion,  but  in  others  it  will  not. 

•  Consider  using  multiple  techniques  to  obtain  estimates 
of  various  sizing  and  timing  parameters. 

•  Be  aware  that  by  ignoring  some  parts  of  the  bounded 

system  that  may  be  very  complex,  but  with  a  low 
probability  of  occurrence,  a  simple  estimate  of  the 
bulk  of  the  system  will  be  better  than  struggling  with 
a  very  complex  portion.  On  the  other  hand,  a  less 
frequently  called  part  of  a  software  program  may  be  the 
most  critical  and,  therefore,  may  have  to  be  addressed 
at  all  costs.  This  will  be  clear  if  the  study 
objectives  and  workloads  have  been  clearly  defined. 

•  Take  advantage  of  other  techniques  that  may  be 

available  to  other  ESD  programs,  such  as  simulator 
packages  or  measurement  packages  that  may  be  under  Air 
Force  lease. 

•  Do  not  ignore  current  literature  on  the  subject  of 

performance  monitoring.  Since  a  system-engineering 
approach  to  the  application  of  performance  analysis  to 
early  estimates  is  in  its  infancy,  new  material  will  be 
constantly  published.  Take  one  or  two  days  to  review 
what  is  currently  being  done. 

•  DO  NOT  LOOK  FOR  MORE  ACCURACY  THAN  IS  POSSIBLE.  AN 

ESTIMATE  IS  STILL  AN  ESTIMATE  NO  MATTER  HOW  IT  IS 
MANIPULATED. 


5.2.4  Identify  and  Qualify  Data  Sources.  Once  the  candidate  techniques 
have  been  selected,  the  sources  of  the  data  required  to  implement  the 
technique  must  be  verified.  Each  technique  contained  in  Tab  A  indicates 
what  types  of  data  must  be  obtained  or  estimated  in  order  to  use  the 
procedure.  When  identifying  and  quantifying  data  sources: 

•  Identify  the  specific  source  of  the  data,  rather  than 
use  a  phrase  such  as,  "to  be  supplied  by  user." 

Identify  WHO  will  provide  it,  WHEN  it  wil.  be  provided, 
and  in  what  FORM  it  will  be  provided. 
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•  Specify  any  unusual  tolerance  limits  that  a  particular 

study  may  require.  For  example,  early  contractor 
software  code  counts  are  usually  estimates  of  the 
total,  and  therefore,  counts  to  the  nearest  1000  may  be 
sufficient.  However,  if  trends  indicate  software 

growth,  demand  the  latest,  most  accurate  data.  1000 
lines  of  code  should  represent  10  modules. 

•  Take  advantage  of  other  management  systems  that  may  be 
generating  data  that  can  be  used.  For  example,  the 
requirements  of  the  Cost/Schedule  Control  Systems 
Criteria  (C/SCSC)  have  been  imposed,  request  the  backup 
that  provides  the  basis  for  calculating  the  Budgeted 
Cost  of  Work  Performed  (BCWP)  in  the  area  of  concern. 

•  Evaluate  the  risk  of  not  having  certain  data,  but  if  a 
worst-case  analysis  indicates  satisfactory  margins 
still  exist,  the  missing  data  is  not  critical  to  the 
study. 

•  Do  not  request  or  obtain  more  data  than  can  be 
analyzed,  either  in  terms  of  time  or  of  the  skill  of 
the  estimator,  unless  steps  are  being  taken  to  acquire 
additional  technical  assistance. 


5.2.5  Prepare  a  Detailed  Study  Approach.  Once  the  objectives  have  been 
set,  the  system  bounded,  candidate  techniques  selected,  and  data  sources 
identified,  a  detailed  study  approach  must  be  developed.  It  should  be 
remembered  that  the  detailed  approach  can  be  anything  from  a  single  page 
to  twenty  pages  or  more,  depending  on  the  complexity  and  requirements  of 
the  estimate.  Even  for  a  small  study,  a  detailed  study  plan  should  be 
prepared.  This  plan  will  form  the  first  part  of  the  study  report.  As  a 
minimum,  the  plan  should  contain  the  topics  listed  in  Table  2  below,  and 
should  be  expanded  as  required. 


Once  the  detailed  study  plan  is  written,  it  must  be  reviewed  to  insure 
that  the  original  objectives  will  be  met. 
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TABLE  2. 


TOPICS  FOR  STUDY  APPROACH 


1.  Tasking  authority 

2.  Objective 

3.  System  boundaries 

4.  Contraints  and  assumptions 

5.  Resources  assigned 

6.  Techniques  that  will  be  used 

7.  Sequence  of  activities 

<3.  A  predetermined  table  in  which  to  enter  results. 


5.2.6  Conduct  the  Sizing  and  Timing  Study.  Conducting  the  sizing  and 
timing  study  is  in  the  same  category  as  acquiring  the  sy-tem  under  study; 
if  everything  goes  as  planned  there  should  be  no  problems,  but  things 
rarely  do.  The  study  team  should  do  the  tasks  outlined  in  the  detailed 
plan,  based  on  the  data  and  objectives.  In  the  event  that  previously 
identified  techniques  and  supporting  data  simply  cannot  be  made  to  work, 
fall  back  to  the  next  level  of  estimating  techniques  and  continue.  This 
will  ca  se  the  anticipated  error  to  be  larger,  but  will  still  provide 
useful  results,  when  conducting  the  study; 

•  3e  alert  for  inconsistent  data.  Such  problems 

d.scusued  prior  to  tae  analysis  of  the  study  results 
will  save  time  and  resources  in  analysis. 

•  Keep  an  organized  record  of  events  during  the  study, 
for  historical  purposes. 

•  Keep  an  open  dialog  among  all  members  of  the  procure¬ 
ment  team  and  users  or  potential  users  of  the  system. 

•  Do  not  become  diverted  from  the  objectives  of  the 
study.  If  a  sudden  requirement  emerges  for  additional 
estimates  during  the  study,  either  begin  again  or 
modify  the  study  accordingly.  In  any  event,  document 
it. 

5.2.7  Correlate  Findings  With  Other  Program  Data.  This  step  can  yield  a 
high  payoff  in  terms  of  total  program  success.  There  is  a  tremendous 
amount  of  data  that  is  either  scheduled  to  be  received  or  is  being 
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received  during  an  acquisition.  At  the  end  of  a  program  all  the  pieces 
should  fit:  total  size,  total  resources  consumed  (including  time',  and 
total  system  performance.  All  these  things  should  relate  during 
development.  If  they  do  not,  the  potential  for  a  problem  exists.  For 

example,  if  the  study  estimate  results  indicate  that  "00,000  lines  of 

source  code  are  required,  the  contractor  estimates  100,000,  and  the 

development  time  could  realistically,  from  historical  data,  support  only 

50,000,  there  exists  a  potentially  critical  problem. 


When  correlating  sizing  and  timing  estimates: 

•  Be  aware  of  built-in  errors  in  the  results. 

•  Anticipate  that  most  unsupported  estimates  are  opti¬ 
mistic. 

•  Demand  backup  and  rationale  for  any  supporting  data. 

•  Make  sure  you  are  relating  similar  units.  Remember  any 
measure  of  system  timing  is  workload  dependent. 


5.2.8  Analyze  Study  Results.  In  analyzing  the  sizing  and  timing  study, 
the  participating  personnel  must  keep  in  mind  the  following: 

•  The  final  results  may  contain  three  types  of  errars; 

1)  erroneous  assumptions,  2)  data  errors,  and  3)  compu¬ 
tational  errors. 

•  Unless  the  study  was  based  completely  on  meusureable 
data  in  a  live  environment-,  the  r°sults  are  only  an 
estimate  and  must  be  presented  as  such. 

•  Sensitivity  analysis  should  be  conducted  to  determine 
the  effect  of  workload  variation,  equipment  pj'ameters, 
and  estimated  data. 

•  Do  not  let  the  analysis  overpower  the  data.  If  the 
objective  of  the  study  is  a  gross  estimate  of  core 
requirements,  the  analysis  should  correspond  to  that 
level  of  effort. 
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5.2.9  Prepare  the  Study  Report.  Preparing  the  sizing  and  timing  study 
report  is  a  key  factor  in  the  process,  since  management  decisions  will 
most  ’ikely  be  made  based  upon  the  results.  In  addition,  the  report  will 
form  the  basis  for  data  collection  and  as  the  source  document  for  updated 
studies  with  comparable  objectives.  Some  guidelines  for  the  preparation 
of  the  report  are: 


•  Keep  it  as  brief  as  possible. 

•  Bound  the  report  to  the  stated  objective.  Any 
information  or  problem  discovered  that  is  worthy  of  an 
additional  study,  or  is  important  to  the  program,  but 
beyond  the  scope  of  sizing  and  timinc,  shouli  be  put  in 
a  separate  appendix  and  brought  to  the  attention  of 
apr-opriate  personnel. 

•  Insure  that  terms  are  fully  described  where  confusion 
could  result  when  later  using  the  report. 

•  Clearly  spell  out  limitations  in  the  use  of  the  data. 

•  If  the  study  is  a  failure,  say  so  and  indicate  why  and 
how,  and  when  it  will  be  redone. 

•  Structure  the  report  to  cover  the  topics  listed  in 
Table  3  below,  at  a  minimum. 


TABLE  3.  TOPICS  FOR  STUDY  REPORT 


[ 

i  1.  Introduction 

2.  Tasking  Authority 

3.  Objective(s) 

4.  Constraints  and  Assumptions 

5.  Bounded  System 

6.  Workload 

7.  Physical  Component  Characteristics 

8.  Techniques  Used 

9.  Conducting  the  Study 

10.  Analysis  of  Results 

11.  Assessment  of  Accuracy 

12.  Other  Program  Implications 

13.  Recommendations  for  the  next  study,  and  corrective  actions! 
to  be  taken,  if  required. 
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5.2.10  Corrective  Action.  corrective  action  is  not  a  function 

of  the  sizing  and  timing  study  personnel  per  se,  it  is  necessary  that  a 
follow-through  exists  whereby  these  individuals  (it  may  be  those  who 

conducted  the  study,  but  with  a  different  title)  who  must  take  action  are 
personally  made  aware  of  the  report.  The  following  guidance  is  provided 
in  the  area  of  corrective  actions: 

•  Involve  all  who  contributed  to  the  report  in  the 
proposed  corrective  actions. 

•  Involve  the  developing  contractor,  appropriate  Program 
Office  personnel,  and  the  user  in  the  proposed 
corrective  action  so  that  in  the  event  the  report  is 
incorrect,  discussions  can  take  place. 

•  Use  the  report  as  a  positive  tool;  most  contractors 
want  to  be  successful,  give  them  a  chance. 

•  Document  any  corrective  actions  taken  and  append  them 

to  the  report.  This  will  assist  future  sizing  and 

timing  study  personnel  in  their  job. 

•  Any  corrective  actions  that  may  have  a  contractual 
impact  must  be  coordinated  with  the  Contracting  Officer. 

All  of  the  above  steps  in  conducting  the  sizing  and  timing  studies  may 
and  should  be  modified  to  meet  the  objectives.  However,  by  structuring 
the  logic  of  the  study  approach,  it  will  be  possible  to  improve  the 
methodology  of  conducting  estimates. 

5 . 3  Study  Update. 

Procedures  for  conducting  sizing  and  timing  study  updates  are 
presented  in  Figure  9  and  are  discussed  in  Sections  5.3.1  through  5.3.7. 

Updating  a  sizing  and  timing  study  must  be  conducted  with  two  goals  in 
mind.  First,  there  must  be  a  striving  for  an  improved  estimate  of  the 
parameter  under  study,  whether  or  not  the  improved  estimate  is  moving  in 

the  desired  direction  (i.e.,  reduced  response  time),  and  second,  the 

amount  of  detail  that  has  become  hard  fact  between  estimates  should  be 
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F i ~ re  9.  :  ani  Timina  Update  Procedures 

Figure  9. 

increasing.  One  danger  in  the  development  of  systems  with  embedded 

computer  systems  is  that  long  periods  of  time  may  pass  with  no  measureable 
improvement  in  sizing  and  timing  parameters.  Initial  contractor  estimates 
that  indicate  no  problems  exist,  are  likely  to  be  accepted  without 
question  until  a  time  when  an  event  that  was  predicted  to  go  smoothly  — 
such  as  subsystem  test — begins  to  indicate  slow  response  time. 

Many  of  the  steps  in  updating  the  sizing  and  timing  study  are  the  same 
as  for  the  initial  study,  but  the  differences  in  some  steps  are 
significant.  The  following  paragraphs  discuss  the  differences  in 
procedures  (see  Figure  7). 

5.3.1  Review  Prior  Study.  When  the  determination  has  been  made  to  update 
a  sizing  and  timing  study,  the  first  step  is  to  review  the  previous  study 
in  order  to  -erify  that  what  is  required  is  truly  an  update  and  not  a  new 
study.  There  can  be  no  hard  and  fast  rules  that  separate  the  update  from 
an  initial  study,  however  the  below  listed  points  should  be  considered: 

UPDATE  IF:  •  The  parameters  being  estimated  are  the  same 

as  a  previous  study. 

and  •  There  existed  a  degree  of  uncertainty  in  the 

previous  study  that  would  result  in  exceeding 
safety  margins  (in  terms  of  storage  and 
t iming) . 
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and  •  There  have  been  no  major  changes  in  the 
configuration  of  the  ECS. 

5.3.2  Verify  Study  Objectives.  The  objectives  for  a  study  update  are 
normally  to  refine  a  previous  study.  There  may  be  a  tradeoff  between 
increased  system  definition,  that  would  reduce  the  anticipated  error,  and 
an  expansion  of  the  objective  scope,  that  may  increase  the  error.  The 
requirement  is  that  an  audit  trail  be  maintained  between  the  two  studi  -s. 

5.3.3  Adjust  System  Boundaries  and  Workload.  The  boundaries  of  the 
previous  system  may  be  expanded,  or  reduced,  as  required  to  support  the 
new  objectives.  There  is  a  distinction  here  between  system  configuration 
and  system  boundary  from  the  study  viewpoint.  The  boundary  is  the  subset 
of  the  ECS  that  has  been  chosen  to  support  the  objective.  The  verif;- 
cation  of  the  workload  must  also  be  adjusted  as  necessary  to  support  the 
updated  requirements  of  the  system.  Multiple  workload  scenarios  may  be 
included  as  the  system  definition  progresses  throughout  the  acquisition. 
The  guidance  provided  in  paragraph  5.2.2  should  be  used  in  verification 
for  the  update. 

3.3.4  Verify  and/or  Modify  Technique(s)  Selection.  This  is  the  area  that 
requires  an  orderly  transition  from  the  classes  of  techniques  that  are 
used  with  limited  data  (Extrapolation)  towards  more  accurate  techniques 
(Measurement)  (see  Figure  6).  The  difficulty  arises  when  long  periods  of 
time  are  allowed  to  transpire  with  no  measureable  improvement  in  system 
definition  or  construction.  In  a  long  development  program:  however,  this 
may  be  a  fact  of  life,  but  there  must  be  a  concerted  effort  to  always  look 
ahead  and  predict  when  the  level  of  knowledge  will  support  a  study 
update.  As  is  presented  in  Section  5.4,  Monitoring,  the  reviewing  of  data 
provided  by  the  contractor  under  the  Contract  Data  Requirements  List 
(CDRL)  can  be  of  assistance  in  determining  when  refined  techniques  may  be 
employed. 


5.3.5  Verify  and/or  Modify  Data  Sources.  The  data  sources  used  in  the 
previous  study  should  be  consulted  for  applicability  to  '.he  study  update. 
Sources  that  proved  to  be  invalid  or  not  available  should  be  reviewed  as 
to  whether  they  have  improved  or  whether  they  should  be  deleted.  New  data 
sources,  which  will  support  improved  techniques,  will  become  available  as 
the  system  progresses.  Care  must  be  taken,  however,  as  in  the  initial 
study,  to  carefully  weigh  the  reliability  of  data  sources.  This  may 
require  the  Program  Office  to  request  backup  data  to  support  any  changes, 
(or  no  change)  in  contractor  estimates  of  code  counts  and  execution 
times.  The  selection  of  data  sources  must  be  fed  back  to  insure  that  they 
will  support  the  techniques  selected. 

5.3.6  Update  Detailed-Study  Approach.  The  previous  detailed-study 
approach  should  be  reviewed  in  two  specific  areas.  First,  does  the 
updated  approach  support  the  new  study  objectives,  and  second,  should  the 
steps  taken  in  the  previous  approach  be  modified  or  eliminated  due  to 
updated  requirements  or  lack  of  usefulness  to  the  earlier  s" udy.  It  is 
expected  that  as  the  skills  of  the  estimating  personnel  increase,  the 
detailed-study  approach  will  become  more  effective. 

5.3.7  Conduct  Study  Update.  Based  on  the  previous  steps,  the  estimating 
personnel  conduct  the  study,  hopefully  more  efficiently.  Detailed  record 
keeping  of  the  conduct  of  the  study  update  is  even  more  important  than 
when  conducting  the  initial  study,  because  the  level  of  detail  of  the 
study  will  be  increasing,  and  it  becomes  m^re  difficult  to  correct  errors 
further  into  the  program.  In  addition,  the  history  of  studies  will 
provide  the  basis  upon  which  to  build  the  overall  skills  of  ESD  personnel 
involved  in  the  estimating  process. 

When  the  study  is  complete,  as  indicated  in  Figure  7,  the  steps  then 
follow  those  specified  for  the  conduct  of  the  initial  studies  as  described 
in  paragraphs  5.2.7  through  5.2.11. 
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5 . 4  Developmental  Monitoring. 

Procedures  for  conducting  sizing  and  timing  developmental  monitoring 
a.-e  presented  in  Figure  10  and  are  discussed  in  Sections  5 . 4 . 1  through 
'>.4.9. 


„  A  O  (  .ABAS,  t  “ 
s  s  :  j  •  a 


rig.  1C.  Sizir.  :  and  Timing  Devt  ic: mental  .'-'.or.  l  tor  inn  F  rr.oiurts 

There  has  been  a  tremendous  amount  of  material  written  and  studied, 
over  the  past  few  years,  on  monitoring  the  development  of  weapon  systems. 
The  primary  management  system  used  throughout  the  Department  of  Def^ ise  is 
the  Cost/Schedule  Control  Systems  Criteria  (C/SCSC)  promulgated  by  Depart¬ 
ment  of  Defense  Instruction  (DODI)  7000.2.  This  system  is  •' n  fact  a  set 
of  criteria  to  be  followed,  and  could  be  imposed  without  receiving  any 
data.  Data  in  support  of  C/SCSC  must  be  requested  on  the  Contract  Data 
Requirements  List  (CDRL)  ,  usually  in  the  form  of  the  Cost  Performance 
Report  (CPR).  This  system,  however,  is  based  upon  dollars  and  manmonths 
as  the  reporting  variables,  and  all  technical  achievement  is  translated 
into  these  terms.  There  also  is  u  great  deal  of  latitude  in  extending  the 
Contract  Work  Breakdown  Structure  (CWBS)  for  a  given  acquisition. 
Although  MIL- STD  881A  is  to  be  used  for  the  project  and  contract  WBS,  the 
extended  >>7BS  is  normally  developed  by  the  contractor  and  approved  by  the 
procuring  agency. 

The  efforts  to  date  in  the  acquisition  of  data  for  C^  systems,  and 
in  fact  all  major  systems  that  have  embedded  computer  systems,  have  been 
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directed  at  developing  resource  measurements  in  terms  marunor.ths , 

computer  time,  and  development  time.  No  significant  effort  has  been  made 
to  collect  functional  system  data  on  sizing  and  timing,  so  therefore,  no 
adequate  sizing  and  timing  data  bases  exist  that  ran  be  used  by  Program 
Offices  or  ESD/TOI.  Obviously  the  measure  of  manmonths  is  unique  and  well 
understood;  however,  system  size  and  measures  of  timing  are  not  agreed 
upon,  or  even  defined  in  a  minimum  number  of  ways.  For  example,  a  single 
numerical  timing  value  for  a  system  is  practically  meaningless  without  the 
specific  description  of  the  system's  workload  and  the  ECS  configuration. 

There  are  five  primary  factors  that  have  contributed  to  the  lack  of 
ECS  developmental  monitoring: 

•  It  is  difficult. 

•  The  data  is  not  available  in  early  stages. 

•  Data  collection  forms  are  inadequate. 

•  C3  systems  are  unique,  hence  workloads  vary  widely. 

•  It  is  time  consuming. 

Developmental  monitoring  as  shown  in  Figure  10  is  started  with  the 
receipt  of  the  first  routine  Data  Item  Description  (DID)  report  submission 
containing  sizing  and  timing  data.  There  is  a  wide  range  of  material, 
from  simple  progress  reports,  to  the  C-5  specification,  and  all  material 
in  between.  In  order  to  bound  this  portion  of  this  report,  one  must 
consider  only  material  that  can  be  cast  in  the  form  of  a  DID  dedicated  to 
sizing  and  timing  data.  The  personnel  involved  in  sizing  and  timing 
estimates,  however,  must  use  technical  data  in  all  forms,  particularly 
with  respect  to  the  step  "Correlate  Findings"  presented  in  paragraph  5.2.7 
above.  All  levels  of  specifications  include  information  that  must  be  used 
in  conducting  sizing  and  timing  studies,  as  well  as  such  DID  generated 
documents  as: 

•  Computer  Program  Development  Plan, 

•  System  (Segment)  Specifications, 

•  Addendum  Specifications, 

•  Product  Specifications, 

•  Technical  Report  (Sizing  and  Timing  Data) , 

•  Computer  Program  Detail  Specification, 
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•  Computer  Program  Flow  Charts,  and 

•  Computer  Program  Sizing  and  Timing  Data. 


There  are  many  more  DXDs  that  generate  sizing  and  timing  data.  In 
fact,  part  of  the  problem  that  has  resulted  from  the  lack  of  success  in 
estimating  and  monitoring  sizing  and  timing  parameters  has  been  the  excess 
of  data  without  any  coherent  order.  Basically,  the  lack  of  monitoring  on 
a  routine  basis  is  more  of  a  contributor  to  disasters  in  the  area  of  ECS 
sizing  and  timing  than  any  other  factor  once  requirements  have  been 
established.  It  is  important  to  stress  that  the  lack  of  monitoring  is 
even  more  critical  than  poor  initial  estimates.  It  is  a  fact  of  life  that 
initial  estimates  are  only  that.  As  such,  initial  estimates  can  not,  and 
should  not,  be  expected  to  be  precise.  This  is  not  to  say  that 
improvement  can  not  be  made,  but  part  of  the  improvement  is  the  acceptance 
that  extensive  design  efforts  must  be  done  before  the  quality  of  a  given 
estimate  is  even  within  50%  in  the  case  of  core  requirements. 

5.4.1  Prepare  Prediction  Set.  The  first  step,  as  indicated  in  Figure  10 
for  developmental  monitoring  is  the  establishment  of  a  Prediction  Set 
form.  The  entire  process  of  the  seven  steps  shown  should  be  completed 
every  month,  with  the  eighth  step  as  soon  as  possible.  This  is  not  only  a 
natural  measure  that  coincides  with  other  program  schedules,  but  relates 
to  the  delivery  of  financial  reports.  A  dedicated  individual  or  group 
should  be  assigned  the  responsibility  for  sizing  and  timing  monitoring. 
The  Prediction  Set  will  be  a  group  of  data  collection  forms  each  with  a 
specific,  measureable  value,  supported  by  constraints.  An  example  of  a 
Prediction  Set  form  is  presented  in  Figure  11. 

The  system  or  subsystem  configuration,  workload,  and  other  constraints 
should  be  specified  in  a  separate  section  of  a  monitoring  file,  maintained 
in  either  manual  or  automated  form,  and  always  retained  for  an  audit 
trail.  When  all  key  values  are  established,  a  master  record  should  be 
constructed  to  provide  for  the  bounded  sizing  and  timing  parameters  being 
tracked  at  a  given  time. 
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PREDICTION  SET  Month  16 

PROGRAM:  KEY  WATCH  DATE:  22  Feb.  ’980 

EVALUATOR:  Lt.  R.  B.  Jones,  USAF 

1 

KEY  VALUE 

(and  Parameters) 

Main  Memory  Ut.lization 
(64K  -  32  Bit/Words) 

i  2 

LAST  VALUE 

Prediction  Set  Month:  15 

of:  25  Jan.  1980 

j 

Estimated  Valje  on  Completion 

57%  Utilization  (36. 5K) 

j  3  |  PREDICTED  VALUE 

|  AS  OF  THIS  DATE 

j 

Estimated  Value  on  Completion 

60%  Utilization  (38. 4K) 

4 

1 

f 

RATIONALE  FOR 

PREDICTION 

Estimated  increase  in  application  j 
software  required  to  produce  two 
new  reports  R-6  and  R-7 

1 

j 

i 

!  5 

i 

1 

i 

| 

i 

i 

CONSTRAINTS 

AS  OF  THIS  DATE 

i 

System  Configuration:  No  Change 
from  Prediction  Set  Month  3 

Workload:  Same  as  Prediction 

Set  Month  5  plus  new  reports  R-6  l 
and  R-7  i 

1 

6 

VALUE  ESTIMATED 

THIS  MONITORING  PERIOD 

59%  utilized  (37. 0K) 

7 

COMMENTS 

{If  Significant  Difference 
Between  Items  3  and  6) 

None 

« 

OTHER  COMMENTS 

Item  6  value  based  on  Contractor's 
estimate  of  increased  lines  of 
source  code  needed  for  reports  R-6 
and  R-7  (see  analysis  in  this 
month's  monitoring  report) 

Figure  11.  Example  of  a  Completed  Prediction  Set  Form 
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5.4.2  Verify  Analysis  Techniques.  For  conducting  monitoring  of  sizing 
and  timing  estimates,  the  same  set  of  techniques  from  Tab  A  are  used. 
Some  of  the  key  factors  in  technique  selection  are: 

•  Try  to  apply  as  many  techniques  as  possible,  even  if 
large  errors  may  exist.  Select  techniques  that  will 
produce  more  accurate  results  if  data  and  funding  are 
available. 

•  Stress  the  evaluation  of  incremental  work  as  it  relates 
to  increased  definition  and  risk  reduction.  That  is, 
probe  to  find  out  how  the  contractor  spent  his  time, 
because  you  may  be  sure  he  spent  money. 

•  Have  the  techniques  selected,  set  up,  or  automated  so 
that  you  may  begin  analysis  as  soon  as  the  monthly  data 
arr ives. 


5.4.3  Receive  Data  or  Request  Update.  On  any  given  program,  close  dialog 
must  exist  with  the  Program  Office's  data  manager  to  evaluate  what  data 
items  contain  sizing  and  timing  data.  Some  steps  to  take  prior  to  and 
after  the  receipt  of  data  are: 

•  Identify  the  documents  to  be  reviewed. 

•  Identify  sizing  and  timing  data  anticipated  from  each. 

•  Develop  a  matrix  to  identify  similar  items  from 

different  data  sources. 

•  Identify  other  Program  Office  technical  personnel  who 

will  be  using  the  data. 

•  When  data  are  received,  note  the  arrival  and  verify 

that  the  sizing  and  timing  information  is  present;  or 
if  not,  determine  why  not. 


5.4.4  Update  Preliminary  Data  File.  All  data  that  affect  sizing  and 
timing  should  be  annotated  and  transferred  to  the  Prediction  Set  form  as 
appropriate  (see  Figure  11).  The  next  step  is  to  perform  required 
analysis.  It  may  be  that  no  analysis  is  required  in  a  particular  area 
because  the  previously  estimated  value  is  still  valid,  such  as  the 
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firmware  in  a  microprocessor.  What  may  happen  at  this  point  is  that 
instead  of  estimating  the  size  of  firmware,  the  emphasis  may  shift  to  the 
performance  of  the  firmware. 

5.4.5  Perform  Analysis.  The  performance  of  sizing  and  timing  monitoring 
analysis  is  really  a  sizing  and  timing  study,  but  on  a  smaller  scale. 

Therefore,  the  steps  from  the  verification  of  the  objective,  through  the 
start  of  report  preparation,  are  the  same.  For  example,  the  item  being 
studied  could  be  the  monitoring  of  the  coding  of  5  modules,  or  about  500 
lines  of  code.  It  has  been  estimated  that  a  programmer  can  normally 
produce  about  100-150  lines  of  complex  code  a  month.  In  the  event  data 

indicates  that  only  50  lines  of  code  were  delivered,  it  may  indicate  the 
programmer  is  encountering  difficulties  and  the  initial  sizing  estimate 
for  the  coding  of  the  5  modules  was  underestimated.  The  analyst  must 

always  be  alert  for  the  indication  of  excessive  delays  in  coding,  that  may 
translate  into  software  size  overruns.  It  also  may  he  that  the  programmer 
coded  300  lines,  and  threw  away  250.  Excessive  code  breakage  is  another 
precursor  of  software  size  growth.  t 

5.4.6  Prepare  Report.  The  report  prepared  as  a  result  of  routine 
monitoring  provides  three  things: 

•  It  forces  routine  reviews  of  selective  sizing  and 
timing  estimates. 

•  It  provides  an  historical  data  base  (although 
unstructured) . 

•  It  provides  the  basis  for  corrective  action,  either  low 
level  or  the  initiation  of  a  more  detailed  sizing  and 
timing  study. 

The  report  should  be  kept  brief,  and  structured  to  be  most  beneficial 
to  the  Program  Office.  The  ro  tine  monitoring  report  shouLd  be  simple, 
and  specific  to  the  acquisition.  For  that  reason  no  elaborate  format  is 
presen  _-d  herein. 
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5.4.7  Update  of  Data  Pile.  When  the  monthly  report  is  complete,  each  of 
the  values  estimated  should  be  entered  as  the  VALUE  ESTIMATED  THIS 
MONITORING  PERIOD  on  the  Prediction  Set  forms  (see  Figure  11)  in  the 
program's  data  file.  This  will  close  the  loop  for  a  given  month  and 
prepare  the  analyst  for  the  next  cycle.  It  may  well  be  that  not  all 
parameters  can  or  need  be  analyzed  every  month.  This  will  be  determined 
by  the  Program  Office's  capability,  size  of  the  acquisition,  amount  of 
changes  impacting  on  sizing  and  timing  parameters,  and  the  risks 
involved.  However,  some  analysis  in  the  areas  of  sizing  and  timing  should 
be  conducted  each  month. 

5.4.8  Take  Corrective  Action.  If  it  appears  that  corrective  action 
should  be  taken,  refer  to  Section  5.2.10. 


6.  INTEGRATION  OF  SIZING  AND  TIMING 


The  increased  emphasis  that  must  be  placed  on  sizing  and  timing  has 
many  facets  woven  throughout  the  acquisition  life-cycle.  All  areas  of 
systems  engineering  have  aspects  that  may  be  affected  by  inadequate  system 
operation  caused  by  the  breaching  of  sizing  and  timing  estimates.  Three 
important  aspects  of  sizing  and  timing  are:  the  discipline  of  sizing  and 
timing;  the  establishment  and  retention  of  spare  memory  capacity;  and 
sizing  and  timing  data  collection. 

6 . 1  Sizing  and  Timing  as  a  Discipline. 

The  discipline  of  sizing  and  timing  estimation  must  be  fit  in  the 
overall  scheme  of  system  acquisition.  There  exists  a  wide  body  of  tech¬ 
nical  knowledge  in  two  general  areas.  Computer  Performance  Evaluation,  and 
Computer  Systems  Performance  Measurement.  For  years,  techniques  such  as 
simulating,  modeling,  benchmarking,  and  monitoring  have  been  used,  but 
primarily  in  the  environment  of  a  data  processing  center  operating  in  a 
batch  or  sequential-job  mode.  Usually  these  techniques  were  applied  to 
optimize  or  fine  tune  an  existing  system.  With  the  recent  explosion  of 
time-sharing  systems  and  distributed  processing,  however,  much  more 
emphasis  is  being  placed  on  systems  that  are  becoming  more  real-time,  like 
C3  systems.  A  recent  example  of  this  increased  emphasis  may  be  seen  in 
the  papers  presented  at  the  Conference  on  Simulation,  Measurement  and 
Modeling  of  Computer  Systems,  held  in  Boulder,  Colorado,  August  13-15, 
1979.  16/ 

Since  estimating  sizing  and  timing  parameters  is  really  a  continuous 
process  for  insuring  that  a  system  performs  effectively,  one  can  recognize 
the  association  with  Verification  and  Validation  (VSV)  activities.  Two  of 
the  Software  Acquisition  Management  Guidebooks  6/,  18/,  that  discuss 

verification  and  validation  of  software,  also  address  sizing  and  timing  at 
the  system  level.  This  recognizes  the  interrelationships  between 
hardware,  software,  firmware,  and  workload.  For  example,  when  one  is 
attempting  to  estimate  the  size  of  the  object  code  for  a  system,  based 
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on.  data  regarding  the  estimated  size  of  the  source  code,  one  must  know  the 
source  to  object  code  conversion  factor  for  the  compiler.  In  certain 
cases  where  storage  capacity  might  be  critical,  one  might  consider  the 
relaxation  of  a  specified  High  Level  Language  (HLL)  for  the  utilization  of 
one  with  a  more  efficient  conversion  factor.  Furthermore,  the  sizes  of 
core  memory  and  necessary  resident  software  and  data  may  dictate  a  need 
for  overlays.  Such  a  need  can  result  in  increased  software  size  and 
reduced  processing  time,  not  to  mention  the  probable  impact  on  future 
modifications.  These  interrelationships  should  be  kept  in  mind  and 

tradeoff  analyses  conducted  when  appropriate  during  the  Conceptual  Phase.  j 

(Also  see  AFR  800-14,  Volume  II,  paragraph  3-4b. ) 

I 

6.2  Spare  Memory  Capacity. 

I 

One  sizing  parameter  that  might  seem  easy  to  obtain,  and  yet  is  one  of  j 

the  most  difficult  parameters  of  the  system  to  define,  is  spare  memory 

capacity.  In  some  instances  this  has  been  defined  as  no  more  than  a 

percent  of  main  memory  (in  thousands  of  object  words)  that  must  be 
reserved  for  future  growth.  This  definition  is  inadequate,  for  it  does 

not  consider  mass  memory  requirements. 

Herd,  et  al  9/  developed  an  algorithm  that  demonstrates  a  relation¬ 
ship  between  core  utilization  and  a  resource  multiplier.  This  relation¬ 
ship  illustrates  that  increased  core  utilizatiin  results  in  increased  cost 
to  develop  a  given  amount  of  software.  From  core  utilization  of  less  than 
60%,  expanding  to  80%,  a  cost  increase  of  up  to  30%  per  instruction  for 

the  total  program  can  be  expected,  and  beyond  80%,  cost  increases  of  up  to  4 

200%  have  been  noted.  Therefore,  it  becomes  critical  to  specify  not  only 

memory  reserve,  but  also  the  reserves  of  all  resources.  A  reserve  of  20% 

is  considered  the  absolute  minimum  and  40%  reserve  would  be  much  more 

appropriate.  Although  AFR  800-14  Volume  II,  paragraph  3-4b  states  that 

reserve  storage  capacity  should  be  available,  it  does  not  specify  the 

amount.  MIL-STD  1679  (Navy)  does  state  that  the  total  system  memory 

capacity  should  contain  at  least  20  percent  reserve  capacity,  at  the  time 

of  acceptance. 
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Acknowledging  that  there  will  probably  be  growth  in  resource  require¬ 
ments  due  to  requirements  changes  and  misestimates,  it  is  recommended  that 
initial  reserves  in  these  categories  be  net  less  than  40%  entering  the 
"’.ill -Scale  Development.  Once  again  it  is  imperative  to  specify  under  what 
conditions  this  minimum  reserve  capacity  will  be  measured,  in  terms  of 
workload.  Close  coordination  must  exist  among  the  personnel  involved  in 
the  generation  of  test  plans  and  procedures  to  insure  that  no  misunder¬ 
standing  exists. 

The  ESD/TOI  guidance  for  estimating  total  memory  utilization  in  order 
to  determine  the  reserve  total  memory  available  is  provided  in  the 
following  equation.  The  reserve  should  equal  one  half  of  the  memory 
capacity  necessary  for  all  the  data  and  programs  required  for  a  system  to 
operate  effectively: 


D  +  P  +  1/2 (D  +  P)  =  Tm 


(Equation  1) 


where 


D 

P 

1/2 (D  +  P) 


Total  number  of  object  words  (in  thousands)  of  data. 
Total  number  of  object  words  (in  thousands)  in  system 
programs. 

Reserve  memory  capacity  (in  this  case  equal  to  33 
percent  of  TM) . 

Total  number  of  object  words  (in  thousands)  (this 
should  equal  the  specified  capacity  of  the  total 
memory) . 


Although  the  above  equation  is  for  estimating  total  memory  capacity  or 
reserve  total  memory,  it  should  also  be  used  to  determine  the  total  core 
or  main  memory  capacity.  In  this  case,  the  total  data,  programs  or 
portions  thereof,  that  must  be  resident  in  the  main  memory  at  one  time, 
should  be  used  in  the  equation. 


6 . 3  Sizing  and  Timing  Data  Collection. 

As  stated  in  Section  5.4,  the  monitoring  of  sizing  and  timing  esti¬ 
mates  is  a  serious  problem.  In  reviewing  Data  Item  Description  (DID) 
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DI-S-30568,  Computer  Program  Timing  and  Sizing  Data,  and  its  application 
to  ESD  projects  (see  Volume  II,  Tab  A1  for  more  information  on  ESD 
projects),  two  points  were  determined: 

•  DI-S-30568  is  not  routinely  applied  to  ESD  acquisitions. 

•  DI-S-30568  is  not  sufficient  for  the  routine  monitoring 
of  sizing  and  timing  estimates. 

It  is  possible  that  DI-S-30568  is  not  being  routinely  applied  because 
of  the  statement  in  paragraph  1.  of  the  DID,  that  states,  "However,  this 
DID  shall  not  unnecessarily  duplicate  descriptive  material  presented  in 
other  documents."  There  could  exist  a  question  as  to  the  meaning  of 
"unnecessarily"  and  "descriptive  material"  as  opposed  to  technical  data. 
In  addition,  no  format  lor  data  display  is  provided,  and  more  importantly, 
no  specific  mention  is  made  of  workload.  Also  there  should  exist  an  audit 
trail  from  page  9  of  the  ESD  Computer  Program  Development  Plan  (CPDP) 
backup  sheet  to  any  routine  reporting  of  sizing  and  timing  parameters. 
Any  DID  used  to  specify  input  documents  must  be  a  viable  document  that 
encompasses  the  entire  system  acquisition  life-cycle  process.  During  the 
Conceptual  Phase,  initial  sizing  and  timing  estimates  and  reserves  should 
be  recorded  by  whatever  activity  is  involved,  and  these  estimates,  and 
subsequent  actual  data,  should  be  retained  through  Full-Scale  Development 
(FSD) .  The  data  will  change  and  possibly  bear  no  relationship  to  initial 
estimates,  but  an  audit  trail  would  be  maintained  if  the  procedures  in 
this  Handbook  are  used. 
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1.  INTRODUCTION 

1 . 1  Considerations  for  Technique  Selection. 

The  selection  of  techniques  for  use  in  estimating  the  sizing  and 
timing  parameters  of  a  computer  system,  should  be  a  function  of  three 
considerations : 

•  the  amount  of  information  known  about  the  system  or 
subsystem  to  be  studied, 

•  the  sizing  and/or  timing  study  objectives,  and 

•  the  number  of  assumptions  that  can  be  made  regarding 
the  system  or  subsystem  and  still  produce  a  reasonable 
estimate. 

1 . 2  Technique  Applications. 

Application  of  techniques  are  discussed  in  Sections  4.3.,  5.2.3, 

5.3.4,  and  5.4.2.  Eighteen  major  techniques  suggested  for  use  are 
presented  in  Figure  A-l  below  and  are  discussed  in  more  detail  in  this 
Tab.  (See  section  1.3  below  for  the  index  of  techniques).  Use  of  these 
techniques  requires  a  great  deal  of  special  experience  and  education.  In 
the  event  ESD  personnel  need  assistance  in  conducting  a  sizing  and  timing 
study,  contact  ESD/TOI.  Note  that  as  techniques  move  from  extrapolation 
to  actual  measurement,  the  expected  reliability  of  each  category  of 
techniques  is  higher.  There  is  no  sharp  distinction  among  three  of  the 
categories,  analytical,  models,  and  simulations,  but  rather,  the  primary 
difference  is  in  the  techniques  and  the  subsequent  reliability  of  the 
results.  Any  specific  technique  may  be  combined  with  another;  for 

example,  models  may  combine  several  analytical  techniques,  or  a 
deterministic  model  may  include  empirical  equations  and  queuing  theory 
equations,  or  else  a  simulation  may  include  both  deterministic  and 

probabilistic  models.  Furthermore,  there  are  times  when  the  use  of  more 
than  one  technique  is  appropriate  in  estimating  different  sizing  and 

timing  parameters  of  a  system.  For  instances,  one  might  conduct  analogies 


Figure  A-l.  Sizing  and  Timing  Techniques 


by  comparing  similar  functions  and  equipments  to  obtain  initial  sizing 
estimates  for  core  or  total  memory  requirements,  and  have  simulations 
performed  to  estimate  the  throughput  and  response  times  of  certain 
subsystems  or  the  entire  system. 

1.2.1  Analogies.  Though  techniques  categorized  as  analogies  are  the 
least  reliable  methods  of  estimating  computer  system  sizing  and  timing 
parameters,  they  are  often  the  only  techniques  that  can  be  applied  in  the 
early  phases  of  the  system  acquisition  due  to  the  lack  of  data  regarding 
the  system  definition.  In  making  an  analogy,  the  more  data  one  has 
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regarding  both  the  new  system  and  the  system  to  which  it  is  being 
compared,  the  better  the  estimate.  In  using  an  analogy,  the  estimator  may 
err  in  the  estimation  of  the  requirements  for  the  new  system  by  not 
considering: 


•  technology  advancements  that  may  have  occurred 
since  the  development  of  the  old  system, 

•  inadequacies  that  exist  in  the  old  system,  such  as 
lack  of  reserve  memory  capacity  or  software 
discrepancies  that  required  modification  during 
the  deployment  phase, 

•  inadequacies  in  the  new  system  specifications  that 
will  result  in  changes,  and 

•  differences  between  the  two  systems  that  will 
impact  on  the  evaluation. 


1.2.2  Analytical  Techniques,  Models,  and  Simulations.  These  categories 
of  techniques  basically  use  the  same  tools  and  only  differ  in  the  degree 
of  complexity.  The  Analytical  group  may  be  considered  appropriate  for 
rough-cut  analysis  of  a  general  system  description  from  the  requirements 
rather  than  any  specific  system.  Use  of  the  Models  group  might  be 
considered  appropriate  when  there  is  a  good  system  definition  and  one  or 
more  hardware  systems  need  evaluation.  The  Simulations  group  performs  the 
imitative  functions  of  a  system  based  on  the  best  knowledge  of  the 
required  functions  and  workload  at  the  time. 

1.2.3  Benchmarks.  The  Benchmarks  group  may  use  the  actual  or  similar 
hardware,  and  may  employ  an  actual  or  synthetic  workload.  The  synthetic 
workload  may  be  based  on  assumed  characteristics  of  the  software,  and  such 
other  assumptions  as  the  amount  of  processing  required  to  complete  certain 
functions  or  jobs,  or  the  amount  of  data  to  be  processed.  Actual  workload 
benchmarks  use  the  actual  software  under  assumed  operating  conditions  and 
are  considered  more  reliable  than  the  synthetic. 


r 
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1.2.4  Monitors.  This  technique  category  is  primarily  used  to  evaluate 
performance  of  completed  or  nearly  completed  operational  systems.  A 
hardware  monitor  collects  data  on  system  usaqe,  utilization,  idle  time, 
etc.,  and  is  transparent  to  the  user,  in  that  it  does  not  occupy  memory, 
nor  does  it  impact  on  the  timing  of  the  sv_>tem.  Data  from  such  a  monitor 
may  be  used  for  sizing  and  timing  studies,  and  also  optimizing  systems. 
In  the  event  a  system  is  approaching  saturation  or  100  percent  utiliza¬ 
tion,  a  hardware  monitor  will  not  indicate  the  amount  of  delay  due  to 
overloading.  The  software  monitor  provides  better  indicators  of 
particular  functions  than  does  the  hardware  monitor.  The  overall 
condition  of  a  system  may  be  estimated  to  be  excellent  through  the 
analysis  of  hardware  monitoring  data;  however,  software  monitoring  data 
may  indicate  that  certain  critical  functions  are  not  performed  within 
prescribed  tolerances  for  specified  operating  conditions.  With  a  software 
monitor,  the  size  of  the  queues  can  be  recorded  and  the  extent  of 
saturation  identified  to  a  greater  degree  than  with  a  hardware  monitor.  A 
disadvantage  of  software  monitors  is  that  they  have  a  sizing  and  timing 
impact  on  the  systems  they  are  monitoring  and  accordingly,  the  monitoring 
data  is  somewhat  distorted.  A  hybrid  monitor  is  a  combination  of  both 
hardware  and  software  monitors,  and  has  the  advantages  of  both.  The 
hybrid  monitoring  data  is  more  complete  than  either  monitor  operating 
separately,  though  it  is  less  reliable  than  the  hardware  monitor  due  to 
the  impact  of  the  software  monitor  portion  of  the  hybrid. 

1 . 3  Tab  A  Index  of  Techniques. 

Sizing  and  Timing  techniques  are  discussed  in  more  detail  on  the 
following  pages  of  this  tab. 


Category  of  Technique  Technique  Page 


Analogy  Similar  Functions  .  A-  7 

Similar  Equipment  .  A-  9 

Experience  and  Judgment  .  .  .  .  A- 11 
Similar  Problem  .  A-12 


Category  of  Technique  Technique  Page 


Analytical  Empirical  Equations  .  A-13 

Algorithms . A- 14 

Piece  Models . A-15 

Boundaries . A-17 

Queuing  Theory . A-18 

Models  Deterministic  .  A-19 

Probabilistic  .  A-20 

Simulations  Deterministic  .  A-21 

Stochastic . A-22 

Benchmarks  Actual  Workload  .  A-23 

Synthetic . A-24 

Monitors  Hardware  .  A-25 

Software  . . A- 26 

Hybrid . A-27 


Monitors  Hardware  .  A-25 

Software  . A- 26 

Hybrid . A-27 


CATEGORY  OF  TECHNIQUE: 


Analogy 


TECHNIQUE: 

DESCRIPTION: 


REQUIRED  DATA: 


ASSUMPTIONS: 


Similar  Functions 


This  technique  assumes  that  a  person  can  esti¬ 
mate  the  sizing  and  timing  parameters  of  a 
proposed  computer  system  by  comparing  identified 
functional  requirements  of  the  proposed  system 
to  those  of  an  existing  computer  system.  This 
technique  may  be  approached  from  two  direc¬ 
tions:  the  top-down  approach  starting  with 

estimates  based  on  a  similar  system  used  in  a 
similar  application;  or  the  bottom-up  approach, 
breaking  the  proposed  system  into  individual 
functions  and  summing  the  results.  In  the 
top-down  approach,  the  estimates  are  refined  as 
more  data  about  the  system  becomes  available. 
In  the  bottom-up  approach,  functions  may  be 
broken  into  subfunctions  to  obtain  more  refined 
estimates. 


Factors  that  should  be  considered,  depending  on 
the  level  of  system  definition,  are  listed  in 
Table  1.  For  the  comparative  system  in  a 
top-down  approach,  the  first  consideration 
should  be  the  system  application.  See  Tab  A  in 
Volume  II  of  this  Handbook  for  examples  of 
recent  ESD  systems.  Next,  specific  computer 
equipment  and  their  characteristics  should  be 
considered.  This  should  be  followed  by  dis¬ 
cussions  with  contractors,  users  and  Program 
Office  personnel  (if  the  office  is  still  in 
existence)  to  obtain  more  specifics,  such  as 
differences  in  functions  between  the  two  systems 
or  the  amount  of  actual  reserve  memory  capa¬ 
city.  Although  the  above  discusses  comparing 
two  systems,  it  does  not  preclude  comparing  the 
proposed  system  to  more  than  one  system  or  a 
number  of  subsystems.  Multiple  comparisons  are 
encouraged  and  should  increase  the  levels  of 
confidence  in  the  developed  estimates. 


The  assumptions  for  both  top-down  and  bottom-up 
approaches  are  similar.  Basically,  each 
approach  assumes  that  systems  that  perform 
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ASSUMPTIONS: 

(continued) 


APPLICATION: 


LEVEL  OF  CONFIDENCE: 


similar  functions  are  similar  in  other  aspects, 
such  as  workoad.  It  is  further  assumed  that 
differences  due  to  expended  functions,  improved 
technology,  or  system  configurations  can  be 
accounted  for  and  estimates  adjusted  accordingly. 


Initial  estimates  of  similar  functions  may  be  in 
gross  figures;  however,  as  more  data  becomes 
available  by  system,  subsystem  or  function,  the 
estimates  should  be  refined.  In  both  the 
top-down  and  the  bottom-up  approaches,  a  point 
should  be  reached  that  produces  relatively 
identical  results. 


One's  level  of  confidence  is  very  dependent  on 
the  quality  of  data  available  and  the  extent  to 
which  the  data  is  analyzed.  Normally,  the  more 
complex  the  systems  are,  the  greater  the  proba¬ 
bility  that  inaccurate  comparisons  will  be  made, 
and  accordingly,  one's  level  of  confidence  will 
decrease. 
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CATEGORY  OF  TECHNIQUE: 


Analogy 


TECHNIQUE: 

DESCRIPTION: 

REQUIRED  DATA: 


ASSUMPTIONS: 

APPLICATION: 


Similar  Equipment 


This  technique  assumes  that  a  person  can 
estimate  the  sizing  and  timing  parameters  of  a 
proposed  computer  system  by  comparing  like 
systems,  while  assuming  that  the  applications  of 
the  systems  are  similar. 


Data  pertaining  to  the  hardware  of  a  system 
being  developed  or  in  existence,  (similar  to  the 
information  presented  in  Tab  A  of  Volume  II  of 
this  Handbook)  should  be  collected  and  compared 
with  similar  data  on  the  proposed  system  hard¬ 
ware.  Not  only  should  equipment  be  compared, 
but  also  the  configurations  of  the  systems, 
since  configuration  differences  may  impact  on 
the  timing  parameters. 


To  assume  that  comparing  similar  equipment  will 
result  in  a  good  correlation  of  sizing  and 
timing  parameters,  one  must  also  assume  that  the 
equipment  will  support  similar  applications  and 
workloads.  Assumptions  regarding  improved 
technology  must  be  considered  and  estimates 
appropriately  revised. 


First  define  the  system  to  be  estimated.  Next, 
examine  and  evaluate  similar  configurations  and 
chose  one  or  more  systems  or  subsystems  that 
approximate  the  application  being  developed. 
Given  special  consideration  to  workload  and  job 
flow  relationships.  Evaluate  impacts  of  con¬ 
figuration  differences,  technology  changes, 
workload  differences,  as  well  as  any  other 
constraints  that  can  be  identified.  Perform 
evaluation  to  determine  sizing  and  timing 
estimates. 
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LEVEL  OP  CONFIDENCE: 


The  level  of  confidence  in  the  results  of  the 
application  of  this  technique  is  very  dependent 
on  the  quality  of  data  available  and  the  extent 
to  which  the  data  is  analyzed.  The  more  complex 
the  systems  are,  the  greater  the  probability  for 
errors,  with  a  resulting  decrease  in  one's  level 
of  confidence. 
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CATEGORY  OF  TECHNIQUE: 


Analogy 


TECHNIQUE: 

DESCRIPTION: 

REQUIRED  DATA: 

ASSUMPTIONS: 

APPLICATION: 


LEVEL  OF  CONFIDENCE: 


Experience  and  Judgment 


This  technique  assumes  that  a  person  can 
estimate  the  sizing  and  timing  parameters  of  a 
proposed  computer  system  by  applying  one's 
experience  with  similar  systems  or  subsystems  to 
more  readily  determine  what  is  needed  to  meet 
the  requirements  of  a  proposed  system. 


Since  the  basis  of  this  technique  is  experience, 
the  data  is  basically  one's  experience  and 
capability  to  recall  specifics  of  systems  or 
subsystems  with  which  one  is  familiar,  or  know 
where  appropriate  sizing  and  timing  data  may  be 
obtained. 


It  is  assumed  that  one's  exposure  to  similar 
systems  or  subsystems  is  adequate  and  reliable. 
The  results  are  dependent  on  judgment. 


The  application  of  this  technique  relies  on 
one's  experience,  and  the  ability  to  recall 
details  of  systems  or  subsystems  one  is  familiar 
with  and  relate  the  knowledge  to  the  proposed 
system.  Based  on  judgment,  one  must  select 
those  systems  or  subsystems  that  closely 
approximate  the  requirements  of  the  proposed 
system.  Next  obtain  and  evaluate  sizing  and 
timing  data  of  the  systems  being  compared  and 
estimate  sizing  and  timing  parameters. 


The  quality  of  a  person's  knowledge  retention 
and  one's  ability  to  recall  or  obtain  data 
determines  the  level  of  one's  confidence  in  the 
results.  Basically,  if  the  person  making  the 
comparison  is  the  one  responsible  for  the 
results,  his  or  her  level  of  confidence  should 
be  higher  than  if  another  person  is  resj  ^nsible 
for  the  results  and  does  not  have  the  experience. 
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CATEGORY  OF  TECHNIQUE: 


Anal  igy 


TECHNIQUE: 

DESCRIPTION: 

REQUIRED  DATA: 

ASSUMPTIONS: 

APPLICATION: 


LEVEL  OF  CONFIDENCE: 


fBSCSBlIKr  PACK  BUNK  -  NOT  IliMCO 


Similar  Problem 


This  technique  assumes  that  a  person  car 
estimate  the  sizing  and  timing  parameters  of  a 
proposed  computer  system  by  comparing  similar 
system  applications  without  singling  out 
specific  software  or  hardware. 


Collection  of  sizing  and  timing  data  pertaining 
to  similar  computer  system  applications. 


It  is  assumed  that  the  computer  systems  or 
subsystems  being  compared  are  similar  in  appli¬ 
cation,  in  many  respects,  and  that  differences 
can  be  identified  and  evaluated  to  determine 
their  impact  on  sizing  and  timing  parameters. 


After  collection  of  sizing  and  timing  data, 
reevaluate  assumptions  regarding  applications, 
and  refine  parameters.  Evaluate  identified 
constraints  on  each  application  and  determine 
relative  impacts.  Especially  consider  the 
relevance  of  differences  in  workload  and,  if 
data  is  available,  differences  in  job  flow. 
Finally,  conduct  an  overall  evaluation  to 
determine  sizing  and  timing  estimates. 


The  level  of  confidence  depends  on  quality  of 
data  and  the  accuracy  of  the  predictions  of 
differences  and  their  relative  impact  on  the 
systems  being  compared. 
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CATEGORY  OF  TECHNIQUE:  Analytical 


TECHNIQUE:  Empirical  Equations 

DESCRIPTION:  Empirical  equations  are  based  on  the  analysis  of 

empirical  data  obtained  by  observation  or 
experience.  The  development  and  use  of 

equations  containing  variables  that  describe 

certain  character  istics  of  a  proposed  computer 
system,  in  order  to  estimate  its  sizing  and 
timing  parameters,  are  based  on  data  developed 
through  simulations  or  actual  systems.  The 

equations  may  range  in  scope  from  predicting  the 
characteristic  of  one  function  of  a  system  to 
predicting  the  overall  characteristics  of  an 
entire  system,  though  the  latter  is  highly 
unlikely  at  this  current  state-of-the-art. 

REQUIRED  DATA:  The  equations  must  be  based  on  data  that  have 

common  characteristics,  to  the  maximum  extent 
possible,  with  the  system,  or  certain  aspects  of 
the  system,  examined.  Particular  attention 

should  be  directed  towards  workload  and  job  flow 
character ist ics. 


ASSUMPTIONS:  It  is  assumed  that  the  empirical  equations  are 

based  on  data  similar  to  the  system  being 
evaluated. 


APPLICATION:  Either  develop  or  obtain  equations  that  use 

characteristics  similar  to  the  system  or 
portions  of  the  system  being  evaluated.  If  the 
equations  are  obtained,  variables  in  the  equa¬ 
tions  may  require  revision  to  relate  to  the 
system  beinq  evaluated.  Once  the  equations  are 
applied,  examine  the  results.  If  possible,  use 
the  equations  with  data  from  actual  systems  to 
determine  accuracy  of  the  equations  and  revised 
variables. 


LEVEL  OF  CONFIDENCE:  One's  level  of  confidence  will  depend  on  the 

results  obtained  from  the  various  equations.  As 
one  becomes  more  familiar  with  certain  equa¬ 
tions,  confidence  may  increase  or  decrease. 


( 


A-13 


CATEGORY  OF  TECHNIQUE: 


Analyt ical 


TECHNIQUE: 

DESCRIPTION: 

REQUIRED  DATA: 

ASSUMPTIONS: 

APPLICATION: 

LEVEL  OF  CONFIDENCE: 


Algor ithms 


The  development  and  use  of  algorithms  employ 
procedures  that  require  iterative  algebraic 
steps  to  obtain  solutions.  Also  see  discussion 
of  algorithms  in  Tab  B  of  Volume  II  of  this 
Handbook. 


The  data  required  will  depend  on  the  specific 
algorithm  or  algorithms  being  applied.  Use 
algorithms  for  which  data  is  available. 


Specific  assumptions  depend  on  the  algorithms 
developed  or  selected  for  use.  Algorithm 
limitations  must  be  strictly  followed. 


Procedures  of  developed  or  selected  algorithms 
must  be  followed.  Data  applied  to  the  algo¬ 
rithms  must  be  defined  exactly  as  specified. 
Constraints  should  be  observed  and  caution  taken 
not  to  extrapolate  results  beyond  allowable 
limits. 


The  level  of  confidence  in  an  algorithm  will 
depend  on  the  accuracy  of  its  development  and 
the  efficiency  with  which  it  is  applied.  One's 
confidence  will  depend  on  results  obtained 
through  use. 


} 


CATEGORY  OF  TECHNI' 

TECHNIQUE: 

DESCRIPTION: 

REQUIRED  DATA: 

ASSUMPTIONS: 

APPLICAT ION: 


}UE:  Analytical 


Piece  Models 


Piece  models  indicate  performance  of  par  t  ~f  a 
system.  Examples  are  modeling  a  buffer,  cr 
number  of  terminals  to  estimate  average  response 
time.  A  piece  model  could  be  the  application  of 
a  model  by  Halstead  8/  that  examines  all 
operations  in  terms  of  operands  and  operators  or 
the  application  of  a  Markov  model  5/. 


The  data  required  depends  on  the  specific  model. 


In  using  a  piece  model  to  represent  a  portion  of 
a  system,  one  must  make  assumptions  about  the 
remainder  of  the  system. 


A  system  can  be  pe  titioned  into  several  sub¬ 
systems  or  processes.  One  or  several  of  these 
processes  may  be  examined  using  piece  models.  A 
system  may  be  represented  as  shown  below: 


Blocks  A,  B,  and  D  may  be  represented  by  piece 
models  and  C  may  have  assumed  characteristics. 
It  is  easiest  to  assume  steady  state  conditions 
for  the  block  C  not  represented  by  a  piece  model. 

As  another  example,  a  Markov  Model  could  be  used 
to  analyze  the  branching  points  of  a  software 
program  to  determine  the  various  paths  of  logic 
flow  and  the  probability  of  the  time  and  use  of 
the  various  paths.  The  results  could  demon¬ 
strate  a  relationship  to  system  workload. 
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LEVEL  OF  CONFIDENCE: 


Confidence  in  any  specific  piece  model  will  vary 
with  the  model,  the  data  on  which  it  is  modeled, 
and  the  complexity  of  the  system  or  subsystem 
being  modeled. 
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CATEGORY  OF  TECHNIQUE: 


Analyt ical 


TECHNIQUE: 

DESCRIPTION: 

REQUIRED  DATA: 

ASSUMPTIONS: 

APPLICATION: 


LEVEL  OF  CONFIDENCE: 


Boundar ies 


This  technique  assumes  that  one  can  obtain 
reasonable  sizing  and  timing  estimates  by 
evaluating  systems  or  processes  within  certain 
boundaries,  such  as  establishing  upper  and  lower 
bounds  on  system  performance  requirements. 


Identify  the  most  utilized  component,  subsystem, 
or  function  in  a  system.  Determine  upper  and 
lower  bounds  for  its  performance. 


It  is  assumed  that  the  most  critically  used 
component,  subsystem,  or  function  sets  the  limit 
on  overall  system  performance. 


First  identify  the  most  critically  used  com¬ 
ponent,  subsystem,  or  function  of  a  system. 
Based  on  other  analytical  methods  or  analogy, 
determine  the  maximum  capability  possible  for 
that  specific  function.  This  would  be  the  upper 
limit.  Then  determine  the  minimum  capability 
that  still  meets  the  established  goals  for  a 
system.  This  would  be  the  lower  limit.  Compare 
these  boundaries  with  the  system  being  evaluated 
to  determine  if  sizing  and  timing  parameters  are 
acceptable. 


One's  level  of  confidence  depends  on  one's 
ability  to  identify  the  most  utilized  component, 
subsystem,  or  function  in  a  system,  and  then  to 
properly  establish  the  upper  and  lower  bounds 
for  analysis. 


CATEGORY  OF  TECHNIQUE: 


Analyt ical 


TECHNIQUE: 

DESCRIPTION: 

REQUIRED  DATA: 

ASSUMPTIONS: 

APPLICATION: 


LEVEL  OF  CONFIDENCE: 


Queuing  Theory 

Queuing  theory  is  a  mathematical  study  of 
waiting  lines  and  encompasses  concepts  of 
service  time,  waiting  time,  and  service  disci¬ 
plines.  This  technique  is  of  value  in  evalu¬ 
ating  critical  points  of  a  system  such  as  data 
waiting  in  a  buffer  until  other  data  in  core  has 
been  processed. 


Processes  within  a  computer  system  are  viewed  as 
lines  for  service.  An  application  of  this 
technique  requires  a  modeling  of  a  system  that 
breaks  it  into  critical  lines  where  the  service 
discipline,  service  times,  and  arrival  rates  are 
either  assumed  or  known. 


The  service  discipline,  service  times,  and 
arrival  rates  must  either  be  assumed  or  known 
for  a  system.  Also,  steady-state  conditions 
must  be  assumed. 


The  systems  or  subsystems  examined  must  be 
reduced  to  lines  with  the  character istics 
specified.  Many  smaller  computer  configurations 
have  already  been  modeled  for  analysis,  and  it 
is  only  necessary  to  assume  the  values  of  the 
parameters  for  each  line.  The  following  authors 
are  but  a  few  who  discuss  queuing  theory  and 
modeling  in  relation  to  computer  systems: 
Ferrari  “/,  Kobayashi  11/,  Martin  13/,  and 
Svobodova  19/. 


Confidence  in  the  results  of  the  application  of 
queuing  theory  will  depend  on  one's  experience 
and  understanding  of  the  technique. 


CATEGORY  OF  TECHNIQUE: 


Models 


TECHNIQUE: 

DESCRIPT  ION: 

REQUIRED  DATA: 

ASSUMPTIONS: 

APPLICATION: 

LEVEL  OF  CONFIDENCE: 


Deterministic 


This  is  in  a  class  of  simple  models  in  which  all 
variables  have  been  determined  rather  than 
obtained  by  random  selection. 


Only  the  factors  specified  are  required  and  no 
knowledge  of  distributions  or  non-steady-state 
situations  is  required. 


It  is  assumed  that  steady-state  conditions 
prevail. 


Identify  the  elements  of  the  system  to  be 
modeled,  find  the  appropriate  models,  and  supply 
the  required  parameters.  The  selection  of  a 
model  depends  on  the  type  of  results  desired. 
For  more  information  regarding  determi nist ic 
models,  refer  to  Ferrari  5/  and  Kobayashi  11/. 


Level  of  confidence  will  depend  on  the  models 
selected  and  one’s  understanding  of  the  models, 
particularly  the  contraints  of  the  selected 
models. 
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CATEGORY  OF  TECHNIQUE: 


Models 


TECHNIQUE: 

DESCRIPTION: 

REQUIRED  DATA: 

ASSUMPTIONS: 

APPLICATION: 

LEVEL  OF  CONFIDENCE: 


Probabilistic 


Probabilistic  models  can  incorporate  statistical 
fluctuations  in  arrival  and  service  demand  rates 
and  times,  in  addition  to  the  simple  parameters 
used  in  deterministic  models.  The  models  are 
useful  in  demonstrating  variations  in  workload. 


The  data  required,  in  addition  to  simple  system 
parameters,  is  a  knowledge  of  the  distribution 
of  arrival  and  service  demand  rates  and  times  of 
a  system  or  subsystem. 


It  is  assumed  that  a  model  approximates  the 
system  being  examined  and  that  the  nature  of  the 
required  statistical  distributions  are  known,  or 
can  be  acquired. 


Identify  the  elements  of  the  system  to  be 
modeled,  find  the  appropriate  models,  and  supply 
the  required  parameters.  The  selection  of  a 
model  depends  on  objectives  of  the  study,  the 
knowledge  of  the  required  statistical  distribu¬ 
tions,  and  the  results  provided  by  selected 
models.  Reference  Ferrari  5/  for  additional 
information. 


With  the  proper  application  of  this  technique, 
one  should  obtain  better  results  than  those 
obtained  from  deterministic  models. 
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CATEGORY  OF  TECHNIQUE: 


Simulations 


TECHNIQUE: 

DESCRIPTION: 


REQUIRED  DATA: 

ASSUMPTIONS: 

APPLICATION: 


LEVEL  OF  CONFIDENCE: 


Deterministic 


A  deterministic  simulation  is  a  numerical 
technique  for  modeling  the  benavior  of  a 
computer  system  without  regard  to  probabilistic 
considerations.  Deterministic  simulations  may 
be  discrete,  continuous,  or  workload  driven. 
Discrete  simulations  are  triggered  by  time.  The 
use  of  Markov  chains  is  an  example  of  a  discrete 
application.  Continuous  simulations  are  driven 
by  events,  but  there  is  no  specific  time 
quantum.  Workload  simulations  are  dependent  on 
workload  conditions. 


The  system  must  be  broken  into  subsystems  that 
can  be  represented  by  a  mathematical  model.  The 
model  is  then  programmed.  The  results  of  the 
program  are  the  character istics  of  the  system. 

It  ;s  assumed  that  the  model  is  a  good  represen¬ 
tation  of  the  system  and  that  steady-state 
conditions  apply. 


The  system  is  first  modeled  or  fitted  to  a  model 
already  available.  The  parameters  are  varied  to 
realize  the  variances  in  results.  The  model  may 
be  run  several  times  with  variations  in  para¬ 
meters  to  yield  a  sensitivity  analysis.  The 
results  are  examined  and  are  compared,  depending 
on  che  assumed  parameters.  The  results  are 
evaluated  and  compared  to  other  simulation 
results,  if  appropriate.  For  additional 
information  regarding  deterministic  simulations, 
refer  to  Ferrari  5/. 


The  results  should  give  a  high  degree  of  confi¬ 
dence,  provided  the  parameters  are  properly 
applied. 
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CATEGORY  OF  TECHNIQUE: 


Simulat ions 


TECHNIQUE: 

DESCRIPTION: 

REQUIRED  DATA: 

ASSUMPTIONS: 

APPLICATION: 


LEVEL  OF  CONFIDENCE: 


Stochast ic 


A  stochastic  or  Monte  Car  .o  simulation  is  a 
mathematical  modeling  of  a  computer  system  that 
includes  characteristics  of  the  system  that  are 
random  in  nature,  such  as  a  workload  that  can  be 
described  by  probability  distributions. 


The  data  required  include  basic  parameters  to 
drive  the  model  and  an  understanding  of  the 
parameters  that  can  be  described  by  probability 
distr ibutions. 


It  is  assumed  that  the  model  and  statistical 
distributions  used  are  refined  approximations  of 
the  system  being  evaluated. 


The  system  is  first  modeled  or  fitted  to  a  model 
already  available.  The  parameters  are  varied  to 
realize  the  variances  in  results.  Also  since 
the  model  generally  will  have  random  variables, 
a  compilation  of  multiple  runs  is  required  for 
each  set  of  variables.  This  will  yield  a  set  of 
statistics  that  better  approximates  the  system. 
The  results  are  examined  and  are  compared 
depending  on  the  assumed  parameters  and  statis¬ 
tical  distributions.  See  Ferrari  5/,  Kobayashi 
11/,  and  Svobodova  20/  for  further  discussions 
on  this  technique. 

Depending  on  the  accuracy  with  which  the 
parameters  are  established,  one  should  have  a 
medium  to  high  degree  of  confidence  in  the 
results. 
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CATEGORY  OF  TECHNIQUE: 


Benchn  arks 


TECHNIQUE: 

DESCRIPTION: 

REQUIRED  DATA: 

ASSUMPTIONS: 

APPLICATION: 

LEVEL  OF  CONFIDENCE: 


Actual  Workload 


Actual  workload  benchmarks  are  used  to  evaluate 
a  system  or  subsystem  using  the  actual  programs 
of  a  computer  system,  under  assumed  conditions. 


A  specification  of  operating  conditions,  work¬ 
load  definition  and  the  system  being  evaluated, 
are  required  before  a  benchmark  can  be  conducted. 


It  is  assumed  that  the  system  or  subsystem  is  in 
existence  and  will  adequately  handle  the  actual 
workload  under  the  assumed  conditions. 


Based  on  the  actual  workload  equirements  and 
using  the  actual  programs,  under  assumed  con¬ 
ditions,  various  performance  and  timing 
parameters  can  be  determined.  The  technique 
does  not  require  an  in-depth  understanding  of 
specific  job  characteristics  since  one  is  using 
actual  workload  requirements.  Reference  Ferrari 
5/,  and  Svobodova  20/  for  more  information. 


Depending  on  approximation  of  the  assumed 
conditions  to  the  actual  conditions,  one  should 
have  a  high  level  of  confidence  in  the  technique 
and  its  results. 
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f. 


CATEGORY  OF  TECHNIQUE: 


Bench/narks 


TECHNIQUE: 

DESCRIPTION: 

REQUIRED  DATA: 

ASSUMPTIONS: 

APPLICATION: 

LEVEL  OF  CONFIDENCE: 


Synthetic 


A  synthetic  benchmark  contains  all  the  important 
ingredients  of  a  s.stem  or  subsystem  except 
certain  characteristics  that  are  developed  for 
the  benchmarking  effort,  such  as  an  estimated 
workload  for  selected  conditions. 


The  parameters  that  might  be  addressed  in  a 
synthetic  benchmark  include  such  characteristics 
as  CPU  processing  demands,  storage  requirements, 
disc  and  file  access  characteristics. 


It  is  assumed  that  only  the  critical  charac¬ 
teristics  need  to  be  modeled  to  obtain  reason¬ 
able  results. 


Identify  critical  characteristics  of  the  system 
or  subsystem  being  evaluated.  Develop  estimated 
characteristics  and  combine  into  an  overall 
system  and  execute.  Ferrari  5/  and  Svobodova 
20/  discuss  the  use  of  this  technique. 


Provided  the  estimated  characteristics  used  are 
reasonable,  this  technique  can  produce  a  medium 
to  high  level  of  confidence  in  its  results. 


CATEGORY  OF  TECHNIQUE: 


Monitors 


TECHNIQUE: 

DESCRIPTION: 

REQUIRED  DATA: 

ASSUMPTIONS: 

APPLICATION: 


LEVEL  OF  CONFIDENCE: 


Hardware 


Hardware  monitors  may  be  either  internal  or 
external  to  a  computer  system  and  are  able  to 
detect  and  collect  very  refined  data  regarding 
performance  characteristics  of  a  system  or 
subsystem. 


Selective  data,  depending  on  the  type  of  monitor 
used,  is  collected  in  a  real-time  mode. 


If  actual  or  simulated  programs  and  workloads 
are  processed,  hardware  monitors  will  provide 
very  refined  data  as  compared  to  software 
monitors. 


Hardware  monitors  are  either  included  in  a 
system's  hardware  or  may  be  interfaced  sub¬ 
sequently.  Since  hardware  monitors  do  not 
directly  affect  the  data  processing  capabilities 
of  a  system,  there  is  no  impact  on  the  results 
obtained  as  is  the  case  with  software  monitors. 
Insure  that  one  understands  how  to  interpret 
results.  For  more  information  regarding  this 
technique,  see  Ferrari  5/  and  Svobodova  20/. 


High  level  of  confidence. 
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CATEGORY  OF  TECHNIQUE: 

TECHNIQUE: 

DESCRIPTION: 


REQUIRED  DATA: 

ASSUMPTIONS: 

APPLICATION: 


LEVEL  OF  CONFIDENCE: 


Monitors 

Software 

Software 

monitors  are 

spec ial 

programs 

that  are 

designed 

to  collect 

data 

about  a 

computer 

system's 

performance. 

Use  of 

software 

monitors 

causes  some  distortion  of  performance  data  due 
to  the  execution  time  required  by  the  software 
monitoring  programs.  Software  monitors  also 
impact  on  sizing  parameters  since  they  require 
memory  space  for  the  program  and  sometimes  the 
data. 


Selective  data,  depending  on  the  software 
monitor's  program,  is  collected  as  the  program 
is  executed. 


If  actual  or  simulated  programs  and  workloads 
are  processed,  software  monitors  will  provide 
fairly  refined  data,  but  not  as  refined  as 
hardware  monitors.  Also,  there  will  be  some 
distortion  of  results  as  mentioned  above. 


Determine  what  sizing  and  timing  parameters  are 
to  be  measured.  Acquire  software  monitors  that 
will  perforin  the  measurements.  Insure  that  one 
has  a  thorough  understanding  of  how  the  moni¬ 
toring  results  are  to  be  interpreted.  Perform 
the  software  monitoring  and  evaluate  the 
results.  In  certain  cases,  some  or  all  of  the 
impacts  of  the  software  monitoring  effort  can  be 
determined  and  the  results  refined  accordingly. 
See  Ferrari  5/  and  Svobodova  20/. 


High  level  of  confidence,  however,  not  as  high 
as  with  hardware  monitors. 


A- 26 


CATEGORY  OP  TECHNIQUE: 


Monitors 


TECHNIQUE: 

DESCRIPTION: 

REQUIRED  DATA: 

ASSUMPTIONS: 

APPLICATION: 

LEVEL  OF  CONFIDENCE: 


Hybrid 


Hybrid  monitors  are  a  combination  of  hardware 
and  software  monitors  that  are  designed  to 
collect  performance  data  from  a  computer  system. 
Certain  parts  of  the  data  collection  may  be 
accomplished  by  either  or  both  the  hardware  or 
software  portions  of  a  hybrid  monitor. 


Selective  data,  depending  on  the  hybrid  monitor 
used. 


If  actual  or  simulated  programs  and  workloads 
are  processed,  hybrid  monitors  will  provide  more 
refined  data  than  will  software  monitors,  though 
not  as  refined  as  hardware  monitors.  Also, 
there  will  be  some  iistcrtion  of  results  due  to 
the  impact  of  the  software  portion  of  the 
monitor. 


Determine  what  sizing  and  timing  parameters  are 
to  be  measured.  Obtain  hybrid  monitors  that 
will  perform  measurements.  Insure  that  one  has 
a  thorough  understanding  of  how  the  monitoring 
results  are  to  be  interpreted.  Perform  the 
monitoring  and  evaluate  the  results.  For  more 
information  on  hybrid  monitors,  see  Ferrari  5/ 
and  Svobodova  20/. 


High  level  of  confidence,  though  not  as  high  as 
hardware  monitors. 
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Program  Objective  Memorandum 
Preliminary  Qualification  Test 


QA  Quality  Assurance 

QAM  Quality  Assurance  Manager 

QAR  Quality  Assurance  Representative 

QDR  Quality  Deficiency  Record 

QQPRI  Quantitative  and  Qualitative  Personnel  Requirements 

Information 


RADC 

RAM 

RJE 

RFP 

ROC 

ROM 

RPG 


Rome  Air  Development  Center 

Random  Access  Memory 

Remote  Job  Entry 

Request  for  Proposal 

Required  Operational  Capability 

Read-Only  Memory 

Report  Program  Generator 


SAMSO  Space  And  Missile  Systems  Organization 

SAMSOP  Space  And  Missile  Systems  Organization  Pamphlet 

SCN  Specification  Change  Notice 

SDR  System  Design  Review 

SOW  Statement  of  Work 

SPO  System  Program  Office 

SQA  Software  Quality  Assurance 

SRR  System  Requirements  Review 

S/W  Software 


TAC  Tactical  Air  Command 

TCTO  Time  Compliance  Technical  Order 

TDD  Top-Down  Design 

TDI  Top-Down  Implementation 

T&E  Test  and  Evaluation 
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J 


TEMP 

TEPT 

TBOA 

TI 

TTY 


VDD 

VS.V 


WBS 

WWMCCS 


Test  and  Evaluation  Master  Plan 
Training  Equipment  Planning  Information 
Test  and  Evaluation  Objectives  Annex 
Technical  Interchange 
Teletypewr iter 


Version  Description  Document 
Validation/Verification 


Work  Breakdown  Structure 

World  Wide  Military  Command  and  Control  System 
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HANDBOOK  GLOSSARY 


The  intent  of  this  glossary  is  to  assist  personnel  in  the  use  of  this 
handbook  only. 


ABSOLUTE  ADDRESS  -  A  memory  address/location  in  a  computer’s  storage  that 
is  specifically  identifier.  See  also  BASE  ADDRESS  and  RELATIVE  ADDRESS. 

ABSOLUTE  ADDRESSING  -  Utilizing  absolute  addresses  in  an  instruction. 

ABSOLUTE  CODING  -  Coding/writing  computer  instructions  that  include 
absolute  addresses. 

ACCEPTABILITY  -  The  degree  to  which  software  or  hardware  meets  the  needs 
of  a  user  or  the  effectiveness  of  the  man-machine  interface. 

ACCESS  TIME  -  The  time  elapsed  between  the  moment  a  request  is  made  for 
data  from  storage  until  the  moment  the  data  is  received. 

ACCUMULATOR  -  A  register  in  the  arithmetic  unit  of  the  computer  which  can 
be  used  to  store  the  results  of  an  operation. 

ACCURACY  -  The  quality  by  which  systems,  programs,  or  data  are  measured  to 
be  error  free. 

ACRONYM  -  A  term  developed  from  the  first  letters  or  selection  of  letters 
in  a  group  of  related  words,  such  as  ESD  for  Electronic  Systems  Division. 

ADAPTABILITY  -  The  ease  with  which  software  or  hardware  can  be  changed  to 
meet  new  or  revised  user  requirements  or  changing  system  environments. 

ADDRESS  -  Character (s)  that  identify  a  location  in  a  storage  device. 

AI/30L  -  An  acronym  for  ALGORITHMIC  LANGUAGE.  A  high-level  computer 
language  designed  to  present  computer  algorithms  in  a  generally  understood 
and  strictly  procedure-oriented  manner. 

ALGORITHM  -  A  definitized  set  of  rules  for  solving  a  problem  in  a  finite 
number  of  steps. 

ALGORITHMIC  LANGUAGE  -  A  language  which  can  present  computer  programs  by 
algor ithms. 

ALLOCATED  BASELINE  -  The  configuration  identification  established  at  the 
end  of  the  requirements/performance  phase. 


GLOSS- 1 


ALPHANUMERIC  -  Pertaining  to  a  set  of  alphabetic,  numeric,  and  symbolic 
characters  in  some  combination. 

ANALOG  COMPUTER  -  A  computer  that  performs  continuously  varying  physical 
processing  of  data  represented  by  nondiscrete  values.  Contrast  to  DIGITAL 
COMPUTER. 

ANALOGY  -  Inference  that  if  two  or  more  items  are  similar  in  some  res¬ 
pects,  they  may  probably  agree  in  others. 

ARRAY  PROCESSORS  -  A  system  composed  of  identical  processing  units 
functioning  under  the  control  of  a  master  CPU.  Same  as  SINGLE  INSTRUCTION 
MULTIPLE  DATA  STREAM  COMPUTERS. 

ASSEMBLE  -  To  create  an  object  language  program  from  a  source  language 
program  by  substituting  machine-oriented  instructions  for  symbolic  machine 
language  instructions. 

ASSEMBLER  -  A  computer  program  used  to  translate  source  coded  instruc¬ 
tions  into  object  coded  instructions.  See  ASSEMBLE. 

ASSOCIATIVE  MEMORY  -  A  storage  device  in  which  locations  are  identified  by 
their  contents  rather  than  by  names  or  addresses. 

ASYNCHRONOUS  COMPUTER  -  A  computer  in  which  each  operation  commences  as  a 
result  of  a  signal  generated  by  the  completion  of  a  previous  operation. 
Contrast  with  SYNCHRONOUS  COMPUTER. 

AUTOMATIC  DATA  PROCESSING  (ADP)  -  The  processing  of  data  by  automatic 
means  with  the  minimum  of  human  intervention. 

AUTOMATIC  DATA  PROCESSING  SYSTQ4  (ADPS)  -  An  assembly  of  automatic  data 
processing  resources  including  equipment,  procedures,  and  communications, 
as  well  as  personnel  in  specified  cases. 

AUTOMATIC  PROGRAMMING  -  Utilizing  a  computer  to  assist  with  certain  seg¬ 
ments  of  program  development. 

AUXILIARY  STORAGE  -  Computer  memory  or  storage  other  than  main  memory, 
such  as  tape  or  disc.  Same  as  SECONDARY  STORAGE. 

AVAILABILITY  -  The  probability  that  a  computer  system  is  operating 
satisfactorily  at  a  specific  time  and  has  the  flexibility  and  capacity  to 
accept  an  additional  workload. 


BACKGROUND  PROCESSING  -  The  machine  execution  of  lower  priority  computer 
programs  when  higher  priority  programs  are  not  utilizing  system  resour¬ 
ces.  Also  see  FOREGROUND  PROCESSING. 
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BACKUP  -  Pertaining  to  additional  or  reserve  system  resources  that  may  be 
utilized  in  the  event  of  problems  with  the  normally  used  sources. 

BASE  ADDRESS  -  The  numeric  value  used  in  combination  with  the  numeric 
value  of  the  relative  address  to  develop  the  absolute  address.  Base 
address  plus  relative  address  equals  absolute  address.  See  also  RELATIVE 
ADDRESS  and  ABSOLUTE  ADDRESS. 

BASELINES  -  A  configuration  identification  document  or  set  of  such  docu¬ 
ments,  and  also  the  computer  system  or  program  itself  after  the  product 
baseline,  formally  designated  and  fixed  at  a  specific  time  during  the 
acquisition  life  cycle.  Baselines,  plus  approved  changes  to  those 
baselines,  constitute  the  current  configuration  identification. 

BASIC  -  An  acronym  for  Beginner's  All-Purpose  Symbolic  Instruction  Code. 
A  simple,  easy  to  learn  high-level  language. 

BATCH  PROCESSING  -  The  processing  of  data  accumulated  over  a  period  of 
time  or  grouped  by  job.  Same  as  SEQUENTIAL  PROCESSING. 

BAUD  -  A  unit  of  measurement  for  signaling  speed  equal  to  the  number  of 
code  elements  per  second. 

BENCHMARKING  -  A  process  of  applying  a  test  workload  on  one  or  more  com¬ 
ponents  of  a  computer  system,  or  one  or  more  computer  systems  to  determine 
specific  capabilities. 

BINARY -CODED  DECIMAL  (BCD)  -  A  binary-coded  notation  consisting  of  decimal 
digits,  made  up  of  four  binary  numerals. 

BINARY  CODE  -  A  code  made  up  of  two  discrete  characters,  usually  0  and  1. 
BINARY  DIGIT  -  See  BIT. 

bit  -  An  acronym  for  Binary  Digit,  that  is  the  most  elementary  unit  of 
information  in  digital  computing,  either  of  the  digits  0  or  1. 

BLOCK  -  A  grouping  of  characters,  words  or  records  that  are  processed  or 
treated  as  an  entity. 

BLOCK  LENGTH  -  The  number  of  characters,  words  or  records  contained  in  a 
block  of  data. 

BOOTSTRAP  LOADER  -  An  input  routine  used  to  initiate  the  loading  of  an 
entire  program  of  which  it  is  a  part. 

BOTTOMS-UP  DESIGN  -  A  traditional  procedure  of  software  development  where 
the  lowest-level  processing  programs  are  coded  first,  module-tested,  and 
made  ready  for  integration  with  additional  programs.  Work  proceeds  in 
this  manner  up  the  hierarchy  of  the  design. 
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BRANCH  -  An  alternative  route  in  a  logic  flow. 

BREAKPOINT  -  A  point  at  which  a  program  may  be  interrupted  for  a  variety 
of  purposes. 

BUFFER  -  A  routine  or  storage  device  used  to  balance  data  rate  flow 
differences  which  occur  with  the  movement  of  data  between  various  devices 
of  a  computer  system. 

BUG  -  An  error  or  flaw  in  a  program  or  equipment  component. 

BULK  STORAGE  -  Large  auxiliary  storage  or  memory.  Same  as  MASS  STORAGE. 

BUS  -  A  circuit  or  path  used  for  data  or  power  transfers  in  a  computer 
system. 

BYTE  -  A  physical  or  logical  grouping  of  binary  digits  usually  operated 
upon  as  a  unit. 


CACHE  MEMORY  -  A  small,  high  speed  storage  device  which  is  imposed  between 

a  processor  and  the  main  memory  in  a  computer  system  to  improve  the 

system's  performance.  Also  see  SEMICONDUCTOR  MEMORY. 

CATHODE  RAY  TUBE  (CRT)  DISPLAY  -  An  input/output  device  which  provides 

visual  displays  of  data,  instructions,  procedures,  etc.,  and  which  usually 
provides  for  user  input  from  a  keyboard  type  of  device. 

CENTRAL  PROCESSING  UNIT  (CPU)  -  The  unit  of  a  computer  system  which 

controls  the  interpretation  and  execution  of  program  instructions.  Same 
as  CENTRAL  PROCESSOR  and  MAIN  FRAME. 

CENTRAL  PROCESSOR  -  See  CENTRAL  PROCESSING  UNIT  (CPU) . 

CHANNEL  -  A  path  along  which  data  is  transferred  between  various  units  or 
components  of  a  computer  system. 

CHARACTER  -  A  letter,  digit  or  symbol  representing  data. 

CHARACTER  RECOGNITION  -  The  reading  and  encoding  of  characters  by  auto¬ 
matic  means. 

CHECK  BIT  -  see  PARITY  BIT. 

CHECK  DIGIT  -  A  digit  used  to  check  for  the  absence  of  errors. 

CLARITY  -  A  measure  of  human  effort  required  to  comprehend  a  program  or 
its  documentation. 
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CLOCK  -  A  device  that  measures  time,  indicates  time,  or  generates  periodic 
signals  in  a  synchronous  computer  to  control  timing  of  operations. 

COBOL  -  An  acronym  for  COmmon  Business  Oriented  Language.  A  high-level 
language  designed  for  commercial  data-processng  problems. 

CODE  -  Representations  or  symbols  for  data,  characters,  text,  etc. 

CODING  -  The  process  of  preparing  a  part  or  all  of  a  program  in  a  high- 
level  or  machine-oriented  language. 

COLLATE  -  To  change  the  arrangement  of  a  group  of  data  or  items. 

COMMAND  AND  CONTROL  (C2)  APPLICATION  -  The  automated  on-board  decision 
making  processes  that  direct  the  operations  of  an  activity  or  vehicle. 

COMMAND,  CONTROL  AND  COMMUNICATIONS  <C3)  SYSTEMS,  AIR  FORCE  -  Encom¬ 
passes  those  systems  required  by  the  Air  Force  to  accomplish  its  defense, 
surveillance,  and  offense  mission  responsibilities.  Such  systems  may  be 
ground  based  (fixed  and  mobile),  airborne,  or  spaceborne;  or  any  combina¬ 
tion  thereof.  Systems  may  also  be  categorized  as  strategic,  tactical,  or 
air  defense  systems. 

COMMUNICATION  APPLICATION  -  The  automated  process  of  transmitting  data 
from  one  location  to  another. 

COMPATIBILITY  -  A  measure  of  interoperability  that  can  be  expected  of 
software  or  hardware  with  other  software  or  hardware  units. 

COMPILE  -  To  prepare  an  object  or  machine-oriented  language  program  from  a 
program  written  in  a  high-level  language. 

COMPILER  -  A  computer  program  designed  to  compile  a  source  language  into 
an  object  language.  Same  as  COMPILING  PROGRAM.  Compare  to  ASSEMBLER. 

COMPILING  PROGRAM  -  See  COMPILER. 

COMPLETENESS  -  Those  attributes  of  software  or  hardware  that  provide  full 
implementation  of  the  functions  required.  Also  see  CORRECTNESS. 

COMPLEXITY  -  The  degree  of  complication  of  a  software  product,  consisting 
of  the  weighted  factor  of  such  measures  as  the  number  of  control  paths, 
number  of  shared  data  references,  number  of  loops,  number  of  interactions 
between  system  components,  user  interfaces,  and  hardware. 

COMPUTER  -  A  data  processor  that  can  perform  a  multitude  of  arithmetic  and 
logic  computations  with  the  minimum  of  human  intervention. 

COMPUTER  INSTRUCTION  -  An  instruction  designed  to  be  recognized  by  the 
Central  Processing  Unit  of  a  computer.  Same  as  MACHINE  INSTRUCTION. 
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COMPUTER  NETWORK  -  Two  or  more  computers  interconnected  through  communi¬ 
cation  links.  Same  as  NETWORK. 

COMPUTER-ORIENTED  LANGUAGE  -  See  MACHINE-ORIENTED  LANGUAGE. 

COMPUTER  PROGRAM  -  A  series  of  instructions  or  statements  in  a  form  that 
may  directly  or  indirectly  be  acceptable  to  a  data  processing  system. 
Also  see  SOFTWARE. 

COMPUTER  PROGRAM  COMPONENT  (CPC)  -  A  functionally  or  logically  distinct 
pact  of  a  computer  program  distinguished  for  purposes  of  convenience  in 
designing  and  specifying  a  complex  computer  program  as  an  assembly  of 
subordinate  elements.  Also  see  FUNCTION. 

COMPUTER  PROGRAM  CONFIGURATION  ITEM  (CPCI)  -  A  specification  that  defines 
the  requirements  for  an  end  item  of  a  computer  program. 

COMPUTER  PROGRAM  DEVELOPMENT  SPECIFICATIONS  -  A  document  which  specifies 
the  total  functional  performance  requirements  for  each  CPCI.  A  specifica¬ 
tion  that  represents  a  comprehensive  and  definitive  statement  of  the  per¬ 
formance,  design,  and  test  requirements  to  be  met  by  a  computer  program. 
Equivalent  to  "Part  I  CPCI  specification"  or  "Type  B5  specifications". 

COMPUTER  PROGRAM  PRODUCT  SPECIFICATION  -  A  document  or  series  of  documents 
which  contain  the  detailed  technical  description  of  the  Computer  Program 
Configuration  Item  (CPCI)  as  designed  and  coded.  It  is  a  complete  des¬ 
cription  of  all  routines,  limits,  timing,  flow,  and  coded  instructions. 
Equivalent  to  "Part  II  CPCI  specification"  or  "Type  C5  specification". 

COMPUTER  RESOURCES  -  The  totality  of  computer  equipment,  computer  programs, 
computer  data,  associated  documentation,  personnel,  and  supplies.  Also 
see  EMBEDDED  COMPUTER  RESOURCES. 

COMPUTER  SOFTWARE  -  A  combination  of  associated  computer  programs  and 
computet  data  required  to  enable  the  computer  equipment  to  perform  data 
processing  or  control  functions.  Also  see  SOFTWARE. 

COMPUTER  SYSTEM  -  See  AUTOMATIC  DATA  PROCESSING  SYSTEM. 

CONCISENESS  -  The  ability  to  satisfy  functional  requirements  with  a  mini¬ 
mum  amount  of  software  or  hardware. 

CONDITIONAL  BRANCH  -  A  logic  flow  transfer  action  which  will  only  occur  if 
certain  specified  conditions  exist  when  the  program  statement  is  exe¬ 
cuted.  Also  see  UNCONDITIONAL  BRANCH. 

CONFIGURATION  -  The  functional  and/or  physical  characteristics  of  software 
or  hardware  as  set  forth  irf  technical  documentation  and  achieved  in  a 
product. 
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CONFIGURATION  CONTROL  -  The  systematic  evaluation,  coordination,  approval 
or  disapproval,  and  implementation  of  all  approved  changes  in  the  con¬ 
figuration  of  a  Configuration  Item  (Cl)  after  formal  establishment  of  its 
configuration  identification. 

CONFIGURATION  ITEM  (Cl)  -  An  aggregation  of  software  or  hardware,  or  any 
of  its  discrete  portions,  which  satisfies  an  end-use  function  and  is 
designated  by  the  Government  for  configuration  management.  CIs  may  vary 
widely  in  complexity,  size,  and  type,  from  a  C3  system  to  a  test  meter. 
Also  see  CRITICAL  ITEM. 

CONFIGURATION  MANAGEMENT  -  The  application  of  technical  and  administrative 
management  direction  and  surveillance  to  accomplish:  the  identification, 
authentication,  and  recording  of  the  functional  and  physical  characteris¬ 
tics  of  a  system;  the  control  of  changes  to  identified  and  authenticated 
characteristics;  and  the  maintenance  of  records  and  issuance  of  reports  on 
configuration  status. 

CONSISTENCY  -  The  degree  to  which  software  or  hardware  satisfies  specifi¬ 
cations,  or  the  extent  that  it  contains  uniform  notation,  terminology,  and 
symbology  within  itself,  and  the  extent  that  the  content  is  traceable  to 
the  requirements, 

CONSOLE  -  A  unit  of  a  computer  used  for  communication  between  a  computer 
and  a  user,  usually  a  system  operator  or  manager. 

CONTRACT  DATA  REQUIREMENTS  LIST  (CDRL)  -  A  contract  form,  DD  1423,  listing 
all  technical  data  items  and  status  data  items,  selected  from  an  Author¬ 
ized  Data  List,  to  be  delivered  under  a  contract. 

CONTROL  FUNCTION  -  The  automated  procedures  involved  in  initiating  an 
action  or  process,  and  subsequently  adjusting  that  action  or  process  based 
on  feedback  data. 

CONVERSATIONAL  MODE  -  A  mode  of  operation  of  a  computer  system  input/ 
output  device  which  allows  a  dialog  between  a  user  and  a  system  during  the 
execution  of  a  program. 

CONVERTER  -  A  device  that  converts  data  from  one  storage  media  to  another, 
such  as  from  tape  to  disc. 

CORE  SIZE  -  The  storage  capacity  of  a  computer  magnetic  core  main  memory, 
which  indicates  the  maximum  number  of  words,  of  a  designated  length,  that 
can  be  stored  at  one  time. 

CORE  STORAGE  -  A  nonmoving  magnetic  storage  unit  that  records  data  or 
programs  by  setting  the  direction  of  magnetization  in  small  toroidal 
shaped  magnetic  material.  Reading  magnetic  core  erases  the  information  so 
that  each  read  must  be  followed  by  a  write  cycle.  Same  as  MAGNETIC  CORE 
STORAGE. 
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CORRECTNESS  -  The  property  of  performing  as  intended  for  all  acceptable 
inputs,  and  the  extent  to  which  software  or  hardware  satisfies  its  speci¬ 
fications  and  fulfills  the  user's  mission  objectives.  Also  see  COMPLETE¬ 
NESS. 

CRITICAL  DESIGN  REVIEW  (CDR)  (COMPUTER  PROGRAM)  -  A  formal  technical 
review  of  the  design  as  depicted  oy  the  specification  and  flow  diagrams, 
sufficiently  detailed  to  enable  a  programmer  to  code,  compile,  and  debug  a 
computer  program,  to  assure  that  design  requirements  have  been  met  before 
beginning  coding. 

CRITICAL  DESIGN  REVIEW  (CDR)  (HARDWARE)  -  A  formal  technical  review  of  the 
design  of  a  machine  item  to  assure  that  design  requirements  have  been 
met. 

CRITICAL  ITEM  -  An  item  within  a  configuration  item  (Cl)  which,  because  of 
special  engineering  or  logistic  considerations,  requires  an  approved 
specification  to  establish  technical  or  inventory  control  at  a  level  below 
the  Cl  level.  The  critical  item  designation  does  not  apply  to  computer 
programs.  Also  see  CONFIGURATION  ITEM. 

CYBERNETICS  -  The  study  of  theories  and  concepts  of  control  and  communi¬ 
cations  in  and  between  living  organisms  and  machines. 

CYCLE  TIME  -  The  minimum  time  interval  between  the  starts  of  successive 
read/write  cycles  of  core  storage. 


DATA  ACQUISITION  SYSTQ1  -  A  system  which  gathers  performance  data  on  a 
computer  program  during  stages  of  development  and  testing,  as  it  performs 
in  a  computer. 

DATA  BASE  -  A  collection  of  data  essential  to  a  computer  system's  opera¬ 
tion. 

DATA  BASE  MANAGEMENT  SYSTEM  (DBMS)  -  A  software  system  which  controls  the 
access,  storage,  updating,  and  maintenance  of  a  data  base. 

DATA  ITEM  DESCRIPTION  (DID)  -  A  standard  form  (DD  Form  1664)  employed  to 
define  format  and  content  requirements  for  specifications,  reports, 
manuals,  and  various  other  items  of  technical  or  management  data  to  be 
delivered  under  a  contract. 

DATA  REDUCTION  -  The  process  of  changing  raw  data  into  a  usable  form. 

DEBUG  -  To  locate  and  eliminate  errors  in  a  program  or  malfunctions  in 
computer  system. 
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DEC IS  ION  TABLE  -  A  table  of  contingencies  related  to  a  specific  problem, 
together  with  the  corresponding  actions  to  be  taken. 

DELAY  LINE  -  A  line  or  network  device  designed  to  reduce  the  speed  at 
which  signal  transmissions  are  made.  Historically  used  as  the  main  memory 
device  of  computers. 

DEPENDABILITY  -  The  probability  that  software  and/or  hardware  will  perform 
in  its  intended  environment. 

DEVELOPMENT  PROCESS  -  A  formal  process  by  which  software  or  hardware 
requirements  are  transformed  into  design  specifications,  created,  or  built 
and  placed  in  operational  status. 

DIGITAL  COMPUTER  -  A  computer  that  performs  arithmetic  and  logical  opera¬ 
tions  on  data  which  is  represented  by  discrete  values.  Contrast  with 
ANALOG  COMPUTER. 

DIRECT  ACCESS  -  The  ability  to  obtain  or  enter  data  directly  from  or  to  a 
computer  storage  location  without  any  prior  data  reference.  Same  as 
RANDOM  ACCESS.  Contrast  with  HIERARCHIAL  MEMORY. 

DISC  STORAGE  -  A  rotating  magnetized  disc  which  can  store  data  or  pro¬ 
grams.  Same  as  MAGNETIC  DISC  STORAGE. 

DISK  -  Variation  of  DISC. 

DISPLAY  FUNCTION  -  The  automated  procedures  involved  in  producing  the 
resultant  type  and  format  of  the  input  process  and  the  output  process  of  a 
system. 

DOCUMENTATION  -  Publications  which  describe  the  design,  logic,  and  opera¬ 
tions  of  software  or  hardware. 

DRUM  STORAGE  -  A  revolving  metal  cylinder  covered  with  a  magnetic 
sensitive  surface  which  can  store  data  or  programs.  Same  as  MAGNETIC  DRUM. 

DUMP  -  To  output/print  the  contents,  data  or  programs,  stored  in  a  storage 
device. 

DUPLEX  -  In  data  communications,  the  capability  to  simultaneously  transmit 
in  two  different  directions  over  a  single  channel. 

DYNAMIC  STORAGE  ALLOCATION  -  A  technique  for  allocating  computer  storage 
space  for  data  and  programs  on  a  controlled,  selective  basis.  A  way  that 
at  one  time  an  item  may  occupy  one  part  of  storage  and  at  another  time  it 
may  occupy  another,  depending  on  the  circumstances. 
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EDIT  -  To  test,  correct  or  modify  data  in  preparation  for  a  subsequent 
operation. 

EFFICIENCY  -  The  degree  to  which  a  task  is  performed  with  a  minimum  con¬ 
sumption  of  time  and  resources.  In  a  computer,  obtaining  maximum  through¬ 
put  with  minimum  execution  time,  storage  space,  and  peripheral  device 
utilization.  Also  see  EXECUTION  EFFICIENCY. 

EMBEDDED  COMPUTER  SYSTEM(S)  (ECS)  -  An  ECS  is  integral  to  an  electronic  or 
electromechanical  system  such  as  in  command,  control,  and  communications 
systems,  combat  weapon  system,  tactical  system,  aircraft,  ship,  missile, 
or  spacecraft  from  a  design,  procurement,  and  operations  viewpoint.  Key 
attributes  include  being  physically  incorporated  into  a  larger  system 
whose  function  is  not  data  processing;  integral  to,  or  supportive  of,  a 
larger  system  from  a  design,  procurement  and  operations  viewpoint;  and 
outputs  include  information,  control  signals,  and  computer  data. 

EMBEDDED  COMPUTED  RESOURCES  -  The  totality  of  computer  equipments,  pro¬ 
grams,  data,  and  communication  links  within  a  larger  system  not  dedicated 
to  data  processing.  Also  see  COMPUTER  RESOURCES. 

EMULATE  -  To  imitate  one  computer  system  by  another  computer  system  so 
that  the  second  system  will  accept  data  and  programs  intended  for  the 
first  system. 

ENVIRONMENT  -  The  conditions  or  circumstances  surrounding  and  influencing 
the  operation  of  a  computer  system. 

ERLANG  -  A  unit  of  measurement  for  traffic  intensity,  measured  as  the 
total  number  of  signals  or  messages  received  during  a  mean  service  time. 

EXECUTION  EFFICIENCY  -  Those  attributes  of  the  software  or  hardware  that 
provide  for  minimum  processing  time.  Also  see  EFFICIENCY. 

EXECUTIVE  SYSTEM  -  See  OPERATING  SYSTEM. 

EXPANDABILITY  -  Those  attributes  of  the  software  that  provide  for  expan¬ 
sion  of  data  storage  requirements  or  computational  functions.  Also  see 
FLEXIBILITY. 

EXTENSIBILITY  -  Extent  to  which  software  or  hardware  can  support  exten¬ 
sions  of  critical  functions. 


FALL-BACK  PROCEDURES  -  Programs  that  operate  in  such  a  way  as  to  circum¬ 
vent  a  fault  that  occurs  in  a  computer  system,  and  which  may  or  may  not 
give  a  degraded  service. 
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FETCH  -  To  obtain  data  from  storage. 


FIELD  -  A  part  of  a  data  record. 

FILE  -  A  collection  of  related  data  records  considered  os  a  unit. 

FILE  MAINTENANCE  -  The  process  of  updating  data  maintained  in  file. 

FIRING  SEQUENCE  CONTROL  FUNCTION  -  The  automated  procedures  involved  in 
directing  the  pattern  of  ignitions  of  missiles,  rockets,  satellite 
directional  jets,  etc. 

FIRMWARE  -  Computer  programs  and  data  loaded  in  a  class  of  memory  that 
cannot  be  dynamically  modified  by  the  computer  during  processing. 

FLEXIBILITY  -  Extent  to  which  a  system  can  absorb  workload  increases  and 
decreases,  or  the  ability  of  a  system  to  immediately  handle  different 
logical  situations.  Also  see  EXPANDABILITY. 

FLOWCHART  -  A  logic  diagram  which  illustrates  the  sequential  processing 
steps  of  a  computer  program  or  computer  system.  Same  as  SYSTEM  FLOWCHART. 

FOREGROUND  PROCESSING  -  The  machine  execution  of  high  priority  computer 
programs  which  preempts  the  use  of  system  resources  by  lower  priority 
programs.  Also  see  BACKGROUND  PROCESSING. 

FORMAL  QUALIFICATION  REVIEW  (FQR)  -  A  review  that  normally  occurs  at 
completion  of  software  validation  testing  to  certify  that  the  test  results 
correspond  to  preestablished  acceptance  criteria.  Successful  completion 
of  the  FQR  establishes  the  Product  Baseline. 

FORMAL  QUALIFICATION  TEST  (FQT)  -  That  portion  of  software  testing  which 
is  conducted  in  accordance  with  approved  test  plans  for  the  purpose  of 
verifying  that  the  software  fulfills  its  requirements.  The  FQT  is  a 
complete  and  comprehensive  test  in  a  continuous  test  period  prior  to 
Functional  Configuration  Audit  (FCA) .  Also  see  FUNCTIONAL  CONFIGURATION 
AUDIT. 

FORTRAN  -  An  acronym  for  FORmula  TRJNslator.  A  high-level  language 
designed  to  facilitate  the  performance  of  mathematical  computations. 

FREQUENCY  OF  OPERATION  -  Measure  of  average  time  between  executions  of  a 
program. 

FUNCTION  -  A  specific  purpose  of  an  entity  of  hardware  or  software  such  as 
what  action  it  will  direct  or  perform.  Also  see  COMPUTER  PROGRAM 
COMPONENT. 

FUNCTIONAL  BASELINE  -  The  initial  system  configuration  identification 
established  at  the  end  of  the  conceptual  phase,  normally  existing  prior  to 
the  start  of  a  software  development  project. 
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FUNCTIONAL  CONFIGURATION  AUDIT  (FCA)  -  A  formal  examination  of  the  test 
data  for  a  Configuration  Item's  functional  characteristics  prior  to 
acceptance,  to  verify  that  the  item  has  achieved  the  performance  specified 
by  its  functional  or  allocated  configuration  identification.  Also  see 
FORMAL  QUALIFICATION  TEST. 

GENERATE  -  To  create  a  machine-or iertted  language  program  from  a  selection 
of  para^ters. 

GENERATOR  -  A  computer  program  that  performs  a  generate  function.  See 
GENERATE. 

GROSCH'S  LAW  -  Predicts  that  computing  power  increases  3S  a  factor  of  the 
square  of  the  cost. 


HARDWARE  -  The  electric,  electronic,  and  mechanical  equipment  used  for 
processing  data  consisting  of  cabinets,  racks,  transistors,  wires,  motors, 
and  such;  or  any  component  of  automatic  data  processing  equipment. 

HARDWARE  ROUTINES  -  Integrated  circuit  logic  units  that  provide  machine 
instructions  for  basic  functions  such  as  multiply,  divide  or  fronting- 
point  conversion  without  the  need  for  software  development. 

HEURISTIC  METHOD  -  An  exploratory  method  of  solving  problems  by  trial  and 
error. 

HEXADECIMAL  NUMBER  SYSTEM  -  A  base  16  number  system  which  uses  the  digits 
0  through  9,  and  the  letters  A  through  F  as  its  symbols. 

HIERARCHIAL  MEMORY  -  A  logical  memory  structure  that  orders  the  programs 
data  and  instructions  so  that  one  leads  to  the  next  within  a  block  of 
memory.  Contrast  with  RANDOM-ACCESS  MEMORY. 

HIGH-LEVEL  LANGUAGE  (HLL)  -  A  machine  independent  language  designed  for 
programming  ease;  must  be  compiled  for  use  with  a  specific  computer.  Also 
see  SOURCE  LANGUAGE.  Same  as  HIGHER-ORDER  LANGUAGE. 

HIGHER-ORDER  LANGUAGE  (HOL)  -  See  HIGH-LEVEL  LANGUAGE  (HLL) . 


HOLLERITH  CODE  -  A  type  of  code  initially  developed  by  Hollerith  for  use 
in  Electrical  Accounting  Machine  (EAM)  cards. 

HOUSEKEEPING  OPERATIONS  -  Operations  that  assist  software  or  hardware  in 
accomplishing  its  functions,  yet  are  not  actually  a  part  the  software  or 
hardware.  Same  as  OVERHEAD  OPERATIONS. 
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IMMEDIATE  ADDRESS  -  The  portion  of  an  address  that  contains  the  operand. 

INDIRECT  ADDRESS  -  An  address  that  identifies  the  location  of  data  to  be 
treated  as  the  address  of  an  operand. 

INPUT/OUTPUT  (I/O)  -  Pertaining  to  a  device  which  perforins  the  input 

process  and  the  output  process  of  a  computer  system. 

INSTRUCTION  -  A  unit  of  logic  in  a  computer  program  that  specifies  one 
operation  and  its  operands. 

INTEGRATED  DATA  PROCESSING  (IDP)  -  Data  processing  that  incorporates  data 
acquisition  and  data  processing  into  a  single  sytem. 

INTEGRITY  -  The  ability  of  one  software  or  hardware  subsystem  to  protect 
the  operation  of  another,  or  a  measure  of  the  degree  of  protection  a 
software  or  hardware  subsystem  or  systems  offers  against  unauthorized 
access  and  loss  due  to  controllable  events. 

INTERFACE  -  A  common  boundary  between  units  of  a  computer  system,  between 
computer  systems,  or  between  a  computer  system  and  another  system. 

INTERFACING  APPLICATION  -  The  automated  processes  that  control  the 
interchange  of  data  between  units  of  a  system  or  between  systems. 

INTERLEAVING  -  A  processing  technique  of  alternating  between  parts  of  two 
or  more  programs,  data,  or  events  while  maintaining  the  identity  of  each. 

INTERPRET  -  To  translate  and  execute  source  language  statements,  one  at  a 
time,  in  sequence. 

INTERRUPT  -  To  suspend  an  ongoing  process  in  order  to  accomplish  a  pre¬ 
determined  function  and  subsequently  may  allow  the  resumption  of  the 
original  ongoing  process.  Often  signalled  by  some  specific  condition  such 
as  input/output  completion,  hardware  errors,  or  some  timing  function. 

ITEM  -  A  portion  of  data  grouping  that  is  considered  as  a  unit. 


JOB  -  A  set  of  data  upon  which  a  computer  operates  that  is  considered  a 
unit  of  work. 

JOB  CONTROL  LANGUAGE  (JCL)  -  A  language  designed  to  identify  a  job  and  the 
job's  requirements  upon  an  operating  system. 
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LABEL  -  The  identification  of  a  set  of  data  or  in  a  program,  an  identifier 
of  an  instruction. 

LASER  -  An  acronym  for  Light  Amplification  by  Stimulated  Emission  of 
Radiation;  a  device  '■hat  produces  an  intense,  coherent,  directional  beam 
of  light. 

LASER  READ-ONLY  MEMORY  -  A  device  that  uses  a  low-power  laser  to  read  a 
data  pattern  in  a  light  sensitive  film  that  has  been  previously  recorded 
by  a  high  power  laser. 

LIBRARY  -  A  collection  of  related  data  files  or  programs;  or  a  storage 
area  for  magnetic  tapes  or  disc  packs. 

LOOKAHEAD  -  A  form  of  processing  in  which  an  instruction  is  fetched  and 
prepared  for  execution  while  a  previous  instruction  is  still  being 
executed . 

LOOP  -  A  set  of  instructions  that  under  specified  circumstances  will  be 
executed  repeatedly. 


MACHINE  INSTRUCTION  -  See  COMPUTER  INSTRUCTION. 

MACHINE-ORIENTED  LANGUAGE  -  A  language  designed  or  designated  for  a 
specific  type  or  class  of  computers.  came  as  COMPUTER-ORIENTED  LANGUAGE. 

MACRO  IN STRUCT ION  -  A  source  language  instruction  that  is  replaced  by  a  set 
of  defined  source  language  instructions. 

MAGNETIC  CORE  STORAGE  -  See  CORE  STORAGE. 

MAGNETIC  DISC  STORAGE  -  See  DISC  STORAGE. 

MAGNETIC  DRUM  -  See  DRUM  STORAGE. 

MAGNETIC  TAPE  STORAGE  -  A  tape  that  is  coated  on  one  side  with  a  magnetic 
sensitive  surface  to  store  data  or  programs.  Same  as  TAPE  STORAGE. 

MAIN  FRAME  -  See  CENTRAL  PROCESSING  UNIT  (CPU) . 

MAIN  MEMORY  -  See  MAIN  STORAGE. 

MAIN  STORAGE  -  The  primary  memory  or  storage  device  in  a  computer  system 
associated  with  the  central  processing  unit  (CPU). 
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MAINTAINABILITY  -  The  extent  to  which  a  software  product  facilitates 
updating  to  correct  errors  and  to  satisfy  new  requirements.  A  maintain¬ 
able  software  product  is  one  which  is  understandable  and  testable  and  can 
be  easily  modified  to  rectify  a  deficiency  and/or  add  new  apahilities. 

MAINTENANCE  -  Changes,  modifications,  restructuring  or  recoding  of  the 
software  for  whatever  reason,  and  is  used  synonymously  with  software 
support. 

MANAGEABILITY  -  The  degree  to  which  a  computer  system  lends  itself  f 
efficient  administration  of  its  components. 

MANAGEMENT  INFORMATION  SYSTEM  (MIS)  -  An  automated  information  system 
designed  to  assist  management  in  decision  making. 

MAP  -  A  listing  of  data  and  programs  with  their  related  locations  in  a 
memory  device. 

MARK  SENSING  -  The  automatic  sensing  of  manually  made  marks  in  an  input 
document. 

MASS  STORAGE  -  See  BULK  STORAGE. 

MASTER  FILE  -  A  relatively  permanent  file  of  data  for  a  specific  job  which 
can  be  updated  if  required. 

MEAN-TIME- BETWEEN- FAILURES  (MTBF)  -  A  determined  average  period  of  time, 
under  specified  conditions,  that  a  functional  unit  will  not  fail  in  an 
assumed  life  of  a  unit. 

MEAN-TIME- TO-REPAIR  (MTTR)  -  A  determined  average  period  of  time  to 
accomplish  the  repair  of  a  functional  unit  in  an  assumed  life  of  a  unit. 

MEGABIT  -  One  million  binary  oits. 

MEGABYTE  -  One  million  bytes. 

MERGE  -  To  combine  two  or  more  sets  of  items  into  one  distinct  set, 
usually  in  some  logical  order. 

M T CPO PROGRAM  -  A  sequence  of  instructions,  hardwired  in  a  computer,  which 
the  computer  uses  to  interpret  machine  language  instructions. 

MICROSECOND  -  One  millionth  ot  a  second,  represented  as  s  or  microsec. 

MILLISECOND  -  One  thousandth  f  a  second,  represented  as  ms  or  msec. 

MISSILE  FIRE  CONTROL  APPLICATION  -  The  automated  processes  that  direct  the 
launching  sequence  for  a  missile. 


GLOSS-15 


MODEM  -  An  interface  device  that  functions  as  a  modulator  and  demodulator 
between  a  computer  system  and  a  communication  link  to  another  system, 
unit,  device  or  sensor. 

MODIFIABILITY  -  A  quality  of  software  or  hardware  that  reduces  the  effort 
required  to  alter  it  in  order  to  conform  to  a  modification  of  its  specifi¬ 
cation. 

MODULE  -  A  set  of  source  instructions  in  a  form  consistent  with  the 
appropriate  language,  and  computer  system  that  encompasses  one  specific 
function  and  has  only  one  entry  statement  and  one  exit  statement.  At  ESD, 
a  module  should  not  exceed  IOC  lines  of  executable  source  code,  excluding 
comments  and  data  definitions. 

MONITOR  -  A  device  or  set  of  routines  that  observe  a  data  processing 
system's  operation  and  identifies  functions  occurring,  probable  problem 
areas,  or  system  performance. 

MULTIPLEX  -  To  simultaneously  use  a  single  channel  of  a  communications 
link  to  transmit  two  or  more  messages. 

MULTIPROCESSOR  -  A  computer  system  with  two  or  more  central  processing 
units  functioning  under  integrated  control. 

MULTIPROGRAMMING  -  The  simultaneous  execution  of  two  or  more  programs  by  a 
central  processing  unit,  usually  effected  by  interleaving  the  programs 
execution  under  control  of  an  operating  system  which  attempts  to  optimize 
overall  performance.. 


NANOSECOND  -  One  billionth  of  a  second,  represented  as  ns  or  nanosec. 

NAVIGATION  APPLICATION  AND  FUNCTION  -  The  automated  processes  involved  in 
determining  the  spatial  position  of  a  vehicle,  such  as  an  aircraft, 
missile,  satellite,  etc.,  and  accomplishing  the  necessary  computations  to 
direct  the  vehicle  towards  another  position. 

NETWORK  -  See  COMPUTER  NETWORK. 


OBJECT  CODE  -  The  output  machine  language  from  an  assembler  or  compiler. 
Same  as  OBJECT  LANGUAGE. 

OBJECT  LANGUAGE  -  See  OBJECT  CODE. 

OBJECT  PROGRAM  -  A  program  assembled  or  compiled  in  object  code. 
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OCTAL  NUMBER  SYSTEM  -  A  base  8  number  system  which  uses  the  digits  0 
through  7  as  its  symbols. 

OFF-LINE  -  Pertaining  to  devices  or  equipment  that  are  not  under  the 
direct  control  of  the  central  processing  unit. 

OFF-LINE  STORAGE  -  Storage  that  is  not  under  the  direct  control  of  the 
central  processing  unit. 

ON-LINE  -  Pertaining  to  devices  or  equipment  that  are  under  the  direct 
control  of  a  central  processing  unit. 

ONLINE  PROCESSING  -  See  ON-LINE. 

OPERAND  -  Data  or  an  address  upon  which  an  operation  is  applied. 

OPERATING  SYSTEM  (0/S)  -  Computer  software  that  directs  the  execution  of 
computer  programs  and  in  some  instances  may  also  supervise  functions,  such 
as  accoun* ing,  assigning  storage,  compiling,  controlling  I/Os,  data 
managing,  debugging,  and  others.  Same  as  EXECUTIVE  SYSTEM. 

OPTICAL  CHARACTER  RECOGNITION  (OCR)  -  A  technique  of  using  a  light- 
sensitive  device  to  read  printed  input  data. 

OVERFLOW  -  The  portion  of  data  that  exceeds  the  prescribed  length  or 
limits  of  a  storage  location. 

OVERHEAD  OPERATIONS  -  See  HOUSEKEEPING  OPERATIONS. 

OVERLAY  -  The  technique  of  utilizing  the  same  areas  of  memory  repeatedly 
during  various  stages  of  the  data  processing  operation.  This  technique 
makes  it  possible  to  execute  programs  that  are  too  large  to  fit  into  the 
main  memory  of  a  computer. 


PACK  -  To  place  data  in  a  compact  form  into  memory  by  employing  certain 
characteristics  of  the  data  and  memory.  Also  see  UNPACK. 

PAGE  -  A  group  of  data  or  instructions,  or  both,  contained  in  a  computer's 
memory,  and  usually  a  specific  size  fixed  by  hardware  design. 

PAGING  -  A  time  sharing  technique  in  which  blocks  of  instructions  or  data 
are  transferred  with  a  computer  system. 

PARALLEX  -  Pertaining  to  concurrent  operations  of  two  or  more  entities. 
Same  as  PARALLEL  PROCESSING. 
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PARALLEL  COMPUTER  -  A  computer  that  can  perform  concurrent  or  parallel 
operations.  Also  see  SERIAL  COMPUTER. 

PARALLEL  PROCESSING  -  See  PARALLEL. 

PARAMETER  MEASUREMENT  FUNCTION  -  The  automated  procedures  involved  in 
determining  distances,  or  quantities  such  as  capacity,  volume,  time, 
speed,  pressure,  temperature,  pulse  rate,  etc. 

PARITY  BIT  -  A  binary  control  digit  attached  to  a  group  of  binary  digits 
so  that  the  resultant  total  is  either  odd  or  even,  dependent  upon 
established  conditions  of  a  particular  system.  Same  as  CHECK  BIT. 

PARITY  CHECK  -  A  test  check  to  determine  if  the  number  of  ones  or  zeros  in 
a  group  of  binary  digits  is  either  odd  or  even  to  determine  if  a  single 
bit  has  been  changed. 

PASS  -  One  complete  cycle  of  processing  a  specified  amount  or  group  of 
data. 

PATCH  -  To  quickly  or  temporarily  correct  a  program  or  computer  system  in 
order  that  it  may  resume  functioning. 

PERFORMANCE  -  Pertaining  to  ability  of  a  computer  system  or  subsystem  to 
perform  its  functions,  measured  in  such  terms  as  response  time,  through¬ 
put,  and  turnaround  time.  These  measures  quantify  the  performance  of  the 
system  or  subsystem  with  respect  to  time  versus  workload. 

PERIPHERAL  EQUIPMENT  -  Computer  system  equipment,  other  than  the  central 
processing  unit  and/or  main  memory. 

PHYSICAL  CONFIGURATION  AUDIT  (PCA)  -  The  formal  examination  of  the  coded 
configuration  of  a  program  element  against  its  technical  documentation  in 
order  to  establish  the  element's  initial  configuration  identification. 

PICOSECOND  -  One  trillionth  of  a  second,  represented  by  ps  or  psec. 

PIPELINING  -  A  form  of  parallel  processing  in  which  a  class  of 
instructions  may  be  simultaneously  executed  in  different  stages  within  a 
processing  unit. 

PL/I  -  See  PROGRAMMING  LANGUAGE  I. 

PORTABILITY  -  The  ability  to  readily  transfer  a  program  from  one  computer 
system  and/or  software  system  environment  to  another. 

PRELIMINARY  DESIGN  REVIEW  (PDR)  -  A  review  that  normally  occurs  at 
completion  of  the  System  Design  Phase.  Successful  completion  of  this 
review  establishes  the  preliminary  computer  system  development  specifica¬ 
tions,  interface  specifications,  and  data  requirements  specifications  in 
the  System  Design  Baseline. 


GLOSS- 18 


PROBLEM -ORi ENTED  LANGUAGE  -  A  high-level  language  designed  for  problem 
solving  such  as  procedure-oriented  languages  or  simulation  languages. 

PROCEDURE-ORIENTED  LANGUAGE  -  A  high-level  language  designed  to  accommo¬ 
date  easy  development  of  algorithmic  procedures. 

PROCESSOR  -  A  device  of  a  system  capable  of  performing  operations  on 
data.  May  be  either  hardware  or  software. 

PRODUCT  BASELINE  -  The  configuration  identification  established  at  end  of 
the  test  and  acceptance  phase. 

PROGRAM  -  A  set  of  procedures  for  accomplishing  solutions  for  particular 
problems. 

PROGRAMMING  FLOWCHART  -  See  FLOWCHART. 

PROGRAMMING  LANGUAGE  I  (PL/I)  -  A  high-level  language  designed  for  busi¬ 
ness  and  scientific  applications. 


QUEUING  THEORY  -  A  field  of  probability  theory  useful  in  analyzing  delays 
at  critical  points  or  nodes  in  a  process  or  some  device  configuration. 


RANDOM  ACCESS  -  See  DIRECT  ACCESS. 

READ-ONLY  MEMORY  (ROM)  -  A  storage  device  which  contains  data  or  programs 
that  can  not  be  inadvertently  erased  or  overwritten  during  the  normal 
operations.  Same  as  SPECIAL-PURPOSE  MEMORY.  See  SEMICONDUCTOR  MEMORY. 

REAL-TIME  -  Pertaining  to  the  accomplishment  of  a  processing  operation 
during  the  same  time  a  related  physical  process  is  occurring  and  which  the 
processing  operation  can  influence.  Same  as  REAL-TIME  PROCESSING,  REAL¬ 
TIME  SYSTEM. 

REAL-TIME  PROCESSING  -  See  REAL-TIME. 

REAL-TIME  SYSTEM  -  See  REAL-TIME. 

RECORD  -  A  group  of  related  data  which  is  treated  as  a  unit. 

RECORDING  DENSITY  -  The  number  of  bits  in  a  single  track  measured  per  unit 
of  length  of  a  recording  medium. 
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REGISTER  -  A  special  purpose  storage  device,  associated  with  the  central 
processing  unit,  for  storing  specified  data. 

RELATIVE  ADDRESS  -  The  numeric  value  used  in  combination  with  the  numeric 
value  of  the  base  address  to  develop  the  absolute  address.  See  also 
ABSOLUTE  ADDRESS  and  BASE  ADDRESS. 

RELIABILITY  -  The  probability  that  software  or  hardware  will  perform  a 
required  function  under  specific  conditions,  without  failure,  for  a 
specified  period  of  time. 

REPORT  GENERATOR  -  A  generator  which  formats  reports  based  on  design 
parameters. 

RESPONSE  TIME  -  The  elapsed  time  between  the  end  of  a  query  input  until 
the  start  of  response  output  for  a  given  workload. 

REUSABILITY  -  Extent  to  which  software  or  hardware  can  be  used  in  other 
applications  or  operations. 

ROBUSTNESS  -  Extent  to  which  software  or  hardware  will  continue  to  perform 
despite  some  violations  of  the  basic  assumptions  in  its  specifications. 

ROUTINE  -  A  set  of  instructions  that  define  a  process. 

RUN  -  The  performance  or  accomplishment  of  a  job  or  operation. 


SECONDARY  STORAGE  -  See  AUXILIARY  STORAGE. 

SEMICONDUCTOR  MEMORY  -  A  solid  state  large-scale  integrated  circuit  that 
has  individual  circuits  which  can  be  set  in  a  conducting  or  nonconducting 
state;  used  in  high  speed  buffers,  read-only  memory,  and  microprogrammable 
memory.  See  CACHE  MEMORY,  READ-ONLY  MEMORY.  Same  as  LSI  MEMORY. 

SEQUENTIAL  PROCESSING  -  See  BATCH  PROCESSING. 

SERIAL  ACCESS  -  The  ability  to  obtain  or  enter  data  into  a  computer 
storage  device  only  in  a  sequential  manner. 

SERIAL  COMPUTER  -  A  computer  which  can  only  perform  operations  sequen¬ 
tially.  Also  see  PARALLEL  COMPUTER. 

SETUP  TIME  -  The  time  required  to  prepare  a  computer  system  for  a  speci¬ 
fied  processing  operation. 

SEXADECIMAL  -  See  HEXADECIMAL. 
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SIGNAL  PROCESSING  APPLICATION  -  The  automated  processes  involved  in 
manipulating  data  prior  to  transmission  or  subsequent  to  receipt. 

SINGLE  INSTRUCTION  MULTIPLE  DATA  STREAM  COMPUTERS  -  See  ARRAY  PROCESSORS. 

SIZE,  COMPUTER  SYSTEM  -  The  size  of  an  embedded  computer  system  has  two 
components:  1)  physical  size;  in  terms  of  the  number  and  types  of  physical 
hardware  units  and  the  number  of  unique  software  entities  including  data 
and  their  corresponding  number  of  source  statements,  object  words,  or 
required  storage  space  as  appropriate;  and  2)  capacity;  in  terms  of 
storage  or  processing  capability  of  hardware  which  is  not  dependent  upon 
software. 

SIZING,  COMPUTER  SYSTEM  -  Activities  involved  in  estimating  the  physical 
and  functional  (configuration)  aspects  of  a  computer  system's  components 
such  as  core  memory,  auxiliary  memory,  software,  virtual  memory,  and  I/O 
devices,  to  determine  its  performance  capabilities  for  processing  data. 

SNAPSHOT  DUMP  -  An  output  report  of  a  selected  storage  area  taken  at 
particular  points  in  time  of  a  program's  execution. 

SOFTWARE  -  Computer  programs  and  in  certain  cases,  associated  documen¬ 
tation.  The  two  types  of  software  are:  (1)  basic  software,  which 
consists  of  programs  designed  to  facilitate  the  use  of  a  particular 
computer  system  and  as  an  operating  system  (0/S)  or  data  base  management 
system  (DBMS) ;  and  (2)  application  software,  which  consists  of  programs 
designed  by  or  for  computer  system  users  to  accomplish  specific  data 
processing  tasks  such  as  command,  control,  and  communications,  weapons 
control,  etc. 

SOFTWARE  ENGINEERING  -  The  science  of  design,  development,  implementation, 
test,  evaluation,  and  maintenance  of  computer  software  over  its  life 
cycle. 

SOLID  STATE  COMPONENT  -  A  component  designed  in  a  solid  physical  state 
such  as  a  transistor. 

SOURCE  CODE  -  See  SOURCE  LANGUAGE. 

SOURCE  LANGUAGE  -  A  computer  program  written  in  a  symbolic  language 
designed  for  programming  ease  which  must  be  translated  into  a  machine- 
oriented  language  to  enable  it  to  be  machine  processable.  Also  see 
HIGH-LEVEL  LANGUAGE  (HLL) ,  HIGHER-ORDER  LANGUAGE  (HOL) ,  SOURCE  PROGRAM. 
Same  as  SOURCE  CODE. 

SOURCE  PROGRAM  -  A  program  written  in  a  high-level  language.  Also  see 
SOURCE  LANGUAGE. 

SPECIAL-PURPOSE  COMPUTE®  -  A  computer  designed  for  special  problems  or 
environments. 
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SPECIAL- PURPOSE  MEMORY  -  See  READ-ONLY  MEMORY  (ROM). 


STORAGE  -  A  storage  device  or  the  capacity  to  hold  data  or  programs  in  a 
computer  system  either  temporarily  or  permanently. 

STORAGE  ALLOCATION  -  The  assignment  of  storage  areas  for  designated  data 
or  programs. 

STORAGE  PROTECTION  -  Protection  built  into  a  computer  system  which  pre¬ 
cludes  reading  and/or  writing  access  to  a  designated  storage  area. 

STRUCTURED  PROGRAMMING  -  A  programming  technique,  based  on  the  mathema¬ 
tically  proven  Structure  Theorem,  which  utilizes  top-down  program  develop¬ 
ment,  programming  support  libraries,  and  chief  programmer  team  concepts. 

SUBROUTINE  -  A  portion  of  a  computer  program  routine  that  performs  a 
specific  or  generalized  function. 

SYNCHRONOUS  COMPUTER  -  A  computer  in  which  each  operation  commences  with 
predetermined  signals  from  a  clock.  Contrast  with  ASYNCHRONOUS  COMPUTER. 

SYSTEM  DESIGN  REVIEW  (SDR)  -  A  review  that  normally  occurs  on  completion 
of  the  System  Design  Phase.  Successful  completion  of  this  review  estab¬ 
lishes  the  preliminary  or  system-level  computer  system  development  speci¬ 
fications,  interface  specifications,  and  requirements  specifications  in 
the  System  Design  Baseline. 

SYSTEM  FLOWCHART  -  See  FLOWCHART. 

SYSTEM  MONITORING  FUNCTION  -  The  automated  procedures  that  evaluate  the 
operational  and  functional  status  of  a  system. 


TABLE  -  A  collection  of  data  arranged  in  such  a  manner  to  facilitate  easy 
reference  during  data  processing  operations,  such  as  a  table  of  terms  or 
values. 

TABLE  UJOK-UP  -  The  act  of  obtaining  a  specified  data  item  in  a  table. 

TAPE  STORAGE  -  See  MAGNETIC  TAPE  STORAGE. 

TARGET  DATA  ENTRY  FUNCTION  -  The  automated  procedures  involved  in  the 
conversion  of  signals  received  by  radar  or  sensors  into  a  form  capable  of 
being  processed  by  an  ESnbedded  Computer  System  (ECS) . 

TARGETT  IDENTIFICATION  FUNCTION  -  The  automated  procedures  involved  in 
processing  data  to  determine  if  a  target  is  friendly  or  hostile. 
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TARGET  TRACKING  FUNCTION  -  The  automated  procedures  involved  in 
determining  the  'patial  position  of  one  or  more  moving  entities  at  regular 
intervals  in  or  ■  r  to  calculate  distance,  course,  speed,  etc. 

TELECOMMUNICATIONS  -  The  transmission  of  messages,  data  or  signals  by 
electronic  means. 

TELETYPEWRITER  (TTY)  -  A  typewriter-like  device  used  to  send  or  receive 
messages. 

TESTABILITY  -  Effort  required  to  test  software  or  hardware  to  insure  it 
performs  its  intended  function. 

THIN  FILM  MEMORY  -  A  storage  device  that  utilizes  a  magnetic  film,  on  a 
thin  glass  plating,  which  is  polarized  for  data  storage. 

THROUGHPUT,  COMPUTER  SYSTEM  -  1)  The  total  production  or  workload  of  a 
computer  system  from  initial  input  of  data,  through  processing,  and 
finally  to  the  output  of  data;  or  2)  the  work  completed  per  unit  time  for 
a  given  workload. 

TIME  SHAPING  -  A  mode  of  operation  that  provides  for  more  than  one  user  to 
utilize  a  single  computer  system  or  units  thereof. 

TIMING,  COMPUTER  SYSTEM  -  The  activities  involved  in  estimating  the  speed 
at  which  a  computer  system  can  handle  a  data  processing  function.  The 
timing  of  a  computer  system  has  two  components:  1)  workload  independent 
parameters  such  as  memory  access  time,  cycle  time,  printer  output  rate, 
etc.  ,  and  2)  workload  dependent  measures  such  as  throughput,  response 
time,  and  utilization. 

TOLERANCE  -  Measure  of  the  ability  of  a  computer  system  to  accept  dif¬ 
ferent  variations  of  the  same  data  as  valid  or  withstand  a  degree  of 
variation  in  input  without  malfunctions  or  rejections. 

TOP-DOWN  DESIGN  (TDD)  -  The  concept  of  hierarchical  design  encompassing 
tiered  levels,  specifications,  and  modules  subordinate  to  the  overall  or 
total  system  level,  such  as  observed  in  tree  diagrams,  break-down  struc¬ 
ture,  and  top-down  structured  programming. 

TOP-DOWN  IMPLEMENTATION  (TOI)  -  The  technique  of  implementing  software 
down  a  hierarchical  structure  which  facilitates  early  testing  of  selected 
segments. 

TRACE  -  A  record  of  how  computer  program  instructions  were  executed. 

TRACEABILITY  -  Those  attributes  of  software  that  provide  continuity  from 
the  requirements  to  the  implementation. 
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TRANSLATOR  -  A  device  or  computer  program  that  translates  programs  in  one 
language  to  another  program  language. 

TUNING  -  Making  minor  modifications  to  system's  hardware,  software  or 
other  aspects  of  a  data  processing  operation  for  the  purpose  of  increasing 
the  efficiency  of  operation. 

TUNING  FUNCTION  -  See  TUNING. 

TURNAROUND  TIME  -  The  time  required  between  the  entry  or  submission  of  a 
job  in  a  computer  system  and  the  receipt  of  the  completed  results. 


UNCONDITIONAL  BRANCH  -  A  branch  action  which  always  occurs  regardless  of 
conditions.  Also  see  CONDITIONAL  BRANCH. 

UNPACK  -  To  restore  the  original  form  of  data  which  had  previously  been 
packed.  Also  see  PACK. 

UPDATE  -  The  action  of  revising  data  in  a  master  file  or  creating  a  new 
master  file  to  reflect  current  conditions,  status  or  corrections. 

UTILITY  PROGRAM  -  A  computer  program  which  provides  general  support  for  a 
computer  system,  such  as  a  sort  or  trace  program. 

UTILIZATION  -  The  ratio  of  time  spent  by  a  compter  system,  component,  or 
device  performing  work  during  a  specified  interval,  to  the  total  time 
available,  at  a  given  workload. 


VALIDATION  -  The  evaluation,  integration,  and  test  activities  performed  at 
the  system  level  to  insure  that  a  computer  system  being  developed  satis¬ 
fies  the  requirements  of  a  System  Specification. 

VARIABLE- LENGTH  RECORD  -  A  record  that  does  not  have  a  specified  length. 

VERIFICATION  -  The  iterative  process  of  determining  whether  the  product  of 
each  step  of  the  Computer  Program  Configuration  Item  (CPCI)  development 
process  fulfills  all  of  the  requirements  specified  by  the  previous  step. 

VIRTUAL  MEMORY  -  See  VIRTUAL  STORAGE. 

VIRTUAL  STORAGE  -  A  technique  which  permits  a  user  to  regard  secondary 
memory  as  an  extension  of  main  memory  and  thus  provides  the  user  with  an 
apparent  larger  main  memory  than  actually  exists.  Same  as  VIRTUAL  MEMORY. 
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WEAPON  FIRE  CONTROL  APPLICATION  -  The  automated  processes  involved  in 
directing  the  positioning  and  firing  sequence  of  one  or  more  weapons  based 
on  target  tracking  data  analysis. 

WORD  -  A  set  of  characters  considered  as  a  unit. 

WORD  LENGTH  -  The  number  of  bits  in  a  word. 

WORK  -  The  sequence  of  operations  required  to  be  performed  by  a  computer 
system  to  obtain  the  specified  results  of  a  designated  amount  of  data 
manipulation. 

WORK  BREAKDOWN  STRUCTURE  (WBS)  -  A  product-oriented  family  tree  composed 
of  hardware,  software,  services  and  other  work  tasks  that  defines  the 
products  to  be  developed  or  produced  and  relates  the  elements  of  work  to 
be  accomplished  to  each  other  and  to  the  end  product. 

WORKING  STORAGE  -  A  section  of  main  storage  designed  for  the  temporary 
storage  of  data  while  it  is  being  used  by  an  operation. 

WORKLOAD  -  The  amount  of  work  performed  by  a  computer  system  with  a  desig¬ 
nated  configuration,  in  a  defined  environment,  and  within  a  specified 
period  of  time.  Also  see  WORK. 


ZERO  SUPPRESSION  -  The  elimination  of  nonsignificant  zeros  from  the  left 
and  right  sides  of  a  numeral. 
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